[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:23 dist-test-slave-x5fp krb5kdc[15002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:23 dist-test-slave-x5fp krb5kdc[15002](info): set up 2 sockets
Aug 10 07:19:23 dist-test-slave-x5fp krb5kdc[15002](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 07:19:25.215752 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.441s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:25.357582 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:45479
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:25.453867 15020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:25.454161 15020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:25.454209 15020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:25.455464 15020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:25.455544 15020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:25.455569 15020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:25.455601 15020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:25.455626 15020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:25.457644 15020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:25.457964 15020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:25.458222 15020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:25.465112 15030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:25.465167 15026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:25.465122 15032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:25.465176 15020 server_base.cc:1034] running on GCE node
I20260810 07:19:25.465602 15020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:25.466076 15020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:25.467276 15020 hybrid_clock.cc:648] HybridClock initialized: now 1786346365467230 us; error 60 us; skew 500 ppm
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:25.474292 15020 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:25.474925 15020 webserver.cc:460] Webserver started at http://127.14.164.62:39243/ using document root <none> and password file <none>
I20260810 07:19:25.475200 15020 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:25.475260 15020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:25.475451 15020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:25.476675 15020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "a01623e304054fdc89aab4c710671daa"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "0b21e8985532400510bc130a3bdf359e"
server_key_iv: "cc0f15ef38e164add2deb4218786e4d8"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.477154 15020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "a01623e304054fdc89aab4c710671daa"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "0b21e8985532400510bc130a3bdf359e"
server_key_iv: "cc0f15ef38e164add2deb4218786e4d8"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.479089 15020 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:25.480212 15044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.480527 15020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:25.480645 15020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "a01623e304054fdc89aab4c710671daa"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "0b21e8985532400510bc130a3bdf359e"
server_key_iv: "cc0f15ef38e164add2deb4218786e4d8"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.480773 15020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:25.491103 15020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:25.493950 15020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:25.494119 15020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:25.500037 15020 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:45479
I20260810 07:19:25.500077 15126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:45479 every 8 connection(s)
I20260810 07:19:25.500579 15020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:25.501855 15127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:25.504451 15127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa: Bootstrap starting.
I20260810 07:19:25.504967 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15020
I20260810 07:19:25.505079 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:25.505280 15127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:25.505604 15127 log.cc:824] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:25.505807 14992 external_mini_cluster.cc:1428] Setting key 210bc2b27f186a2f3a96392011f51fb4
I20260810 07:19:25.506399 15127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa: No bootstrap required, opened a new log
I20260810 07:19:25.508929 15127 raft_consensus.cc:348] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a01623e304054fdc89aab4c710671daa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45479 } }
I20260810 07:19:25.509142 15127 raft_consensus.cc:374] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:25.509181 15127 raft_consensus.cc:729] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01623e304054fdc89aab4c710671daa, State: Initialized, Role: FOLLOWER
I20260810 07:19:25.509312 15127 consensus_queue.cc:260] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a01623e304054fdc89aab4c710671daa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45479 } }
I20260810 07:19:25.509473 15127 raft_consensus.cc:388] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:25.509516 15127 raft_consensus.cc:482] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:25.509560 15127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:25.510818 15127 raft_consensus.cc:504] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a01623e304054fdc89aab4c710671daa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45479 } }
I20260810 07:19:25.510955 15127 leader_election.cc:302] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a01623e304054fdc89aab4c710671daa; no voters:
I20260810 07:19:25.511149 15127 leader_election.cc:288] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:25.511217 15135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:25.511384 15135 raft_consensus.cc:686] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [term 1 LEADER]: Becoming Leader. State: Replica: a01623e304054fdc89aab4c710671daa, State: Running, Role: LEADER
I20260810 07:19:25.511467 15127 sys_catalog.cc:564] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:25.511492 15135 consensus_queue.cc:237] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a01623e304054fdc89aab4c710671daa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45479 } }
I20260810 07:19:25.511827 15136 sys_catalog.cc:455] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a01623e304054fdc89aab4c710671daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a01623e304054fdc89aab4c710671daa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45479 } } }
I20260810 07:19:25.511960 15136 sys_catalog.cc:458] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:25.512166 15138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a01623e304054fdc89aab4c710671daa. Latest consensus state: current_term: 1 leader_uuid: "a01623e304054fdc89aab4c710671daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a01623e304054fdc89aab4c710671daa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45479 } } }
I20260810 07:19:25.512260 15138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:25.512241 15143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:25.513018 15143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:25.514887 15143 catalog_manager.cc:1347] Generated new cluster ID: 4c2b3acf6d404fc395b483879d611f41
I20260810 07:19:25.514952 15143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:25.526158 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:25.506877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.14.164.62:45479)
0810 07:19:25.507132 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:25.507142 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:25.507329 (+ 187us) server_negotiation.cc:407] Connection header received
0810 07:19:25.509964 (+ 2635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:25.509973 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:25.510030 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:25.510148 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:25.512682 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.514170 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.515788 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.516460 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.516587 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:25.521487 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:25.521493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:25.521493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:25.521508 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:25.523709 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:25.524331 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:25.524334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:25.524335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:25.524464 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:25.524832 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:25.524833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:25.524833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:25.525181 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:25.525330 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:25.525682 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 07:19:25.525832 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":118,"threads_started":1}
I20260810 07:19:25.551676 15143 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:25.552278 15143 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:25.562455 15143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a01623e304054fdc89aab4c710671daa: Generated new TSK 0
I20260810 07:19:25.562717 15143 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:25.604369 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45479
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:25.692787 15166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:25.693002 15166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:25.693028 15166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:25.694193 15166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:25.694239 15166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:25.694267 15166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:25.695726 15166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45479
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:25.695938 15166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:25.696127 15166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:25.701478 15176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:25.701484 15175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:25.701484 15178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:25.701766 15166 server_base.cc:1034] running on GCE node
I20260810 07:19:25.702013 15166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:25.702275 15166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:25.703449 15166 hybrid_clock.cc:648] HybridClock initialized: now 1786346365703435 us; error 29 us; skew 500 ppm
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:25.706271 15166 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:25.706789 15166 webserver.cc:460] Webserver started at http://127.14.164.1:41535/ using document root <none> and password file <none>
I20260810 07:19:25.707073 15166 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:25.707149 15166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:25.707305 15166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:25.708413 15166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "c0f78bf495204f85895dba80a972971c"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "251486658405f07bc407b78034308338"
server_key_iv: "302bf20ef77b3a6c975e5d1ae4d00413"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.708899 15166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "c0f78bf495204f85895dba80a972971c"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "251486658405f07bc407b78034308338"
server_key_iv: "302bf20ef77b3a6c975e5d1ae4d00413"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.710382 15166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:25.711333 15188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.711755 15166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:25.711881 15166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "c0f78bf495204f85895dba80a972971c"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "251486658405f07bc407b78034308338"
server_key_iv: "302bf20ef77b3a6c975e5d1ae4d00413"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.711987 15166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:25.722199 15166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:25.724473 15166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:25.724664 15166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:25.724963 15166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:25.725322 15166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:25.725382 15166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.725426 15166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:25.725455 15166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.732239 15166 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:42937
I20260810 07:19:25.732295 15377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:42937 every 8 connection(s)
I20260810 07:19:25.732668 15166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:25.740882 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15166
I20260810 07:19:25.741003 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:25.741189 14992 external_mini_cluster.cc:1428] Setting key 0f3eac4fae2fda51ee2d9daa1e1aa912
I20260810 07:19:25.744395 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:25.733658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:42997 (local address 127.14.164.62:45479)
0810 07:19:25.733749 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:25.733751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:25.733977 (+ 226us) server_negotiation.cc:407] Connection header received
0810 07:19:25.734487 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:25.734488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:25.734499 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:25.734562 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:25.735702 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.736219 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.737351 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.737567 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.737646 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:25.740218 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:25.740219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:25.740220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:25.740228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:25.742406 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:25.743096 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:25.743097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:25.743097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:25.743203 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:25.743518 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:25.743519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:25.743520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:25.743704 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:25.743789 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:25.744173 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 07:19:25.744285 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:19:25.744707 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:25.733616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45479 (local address 127.14.164.1:42997)
0810 07:19:25.733793 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:25.733882 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:25.734401 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:25.734619 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:25.734622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:25.735362 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:25.735593 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:25.735599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.736331 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.736332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:25.737237 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:25.737240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.737703 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.737705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:25.737839 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:25.738545 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:25.738556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:25.740104 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:25.742499 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:25.742529 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:25.742531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:25.742929 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:25.743291 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:25.743292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:25.743292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:25.743440 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:25.743811 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:25.743819 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:25.743981 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 07:19:25.744142 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:19:25.744369 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260810 07:19:25.745461 15378 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45479
I20260810 07:19:25.745601 15378 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:25.745843 15378 heartbeater.cc:511] Master 127.14.164.62:45479 requested a full tablet report, sending...
I20260810 07:19:25.746572 15074 ts_manager.cc:194] Registered new tserver with Master: c0f78bf495204f85895dba80a972971c (127.14.164.1:42937)
I20260810 07:19:25.747730 15074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:42997
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:25.777803 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45479
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:25.866371 15384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:25.866580 15384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:25.866604 15384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:25.867744 15384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:25.867794 15384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:25.867822 15384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:25.869360 15384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45479
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:25.869603 15384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:25.869848 15384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:25.875397 15395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:25.875514 15393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:25.875622 15384 server_base.cc:1034] running on GCE node
W20260810 07:19:25.875442 15392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:25.875947 15384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:25.876184 15384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:25.877337 15384 hybrid_clock.cc:648] HybridClock initialized: now 1786346365877323 us; error 29 us; skew 500 ppm
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:25.880100 15384 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:25.880595 15384 webserver.cc:460] Webserver started at http://127.14.164.2:35645/ using document root <none> and password file <none>
I20260810 07:19:25.880913 15384 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:25.880986 15384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:25.881145 15384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:25.882266 15384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "39688f27d8334debaa768ceba83f1ce9"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "4e1043f6def1e65ee72c6443ee3793ce"
server_key_iv: "065beb8d03324893a1b9e0bc4125c66a"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.882663 15384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "39688f27d8334debaa768ceba83f1ce9"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "4e1043f6def1e65ee72c6443ee3793ce"
server_key_iv: "065beb8d03324893a1b9e0bc4125c66a"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.884173 15384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:25.885159 15409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.885462 15384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:25.885592 15384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "39688f27d8334debaa768ceba83f1ce9"
format_stamp: "Formatted at 2026-08-10 07:19:25 on dist-test-slave-x5fp"
server_key: "4e1043f6def1e65ee72c6443ee3793ce"
server_key_iv: "065beb8d03324893a1b9e0bc4125c66a"
server_key_version: "encryptionkey@0"
I20260810 07:19:25.885680 15384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:25.897943 15384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:25.900820 15384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:25.901003 15384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:25.901345 15384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:25.901768 15384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:25.901819 15384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.901858 15384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:25.901893 15384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:25.909603 15384 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:43159
I20260810 07:19:25.909644 15579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:43159 every 8 connection(s)
I20260810 07:19:25.910022 15384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:25.914453 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15384
I20260810 07:19:25.914569 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:25.914759 14992 external_mini_cluster.cc:1428] Setting key 643a69dcf4dbcc74cd064e69c41db9e4
Aug 10 07:19:25 dist-test-slave-x5fp krb5kdc[15002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:25.921954 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:25.910948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:47207 (local address 127.14.164.62:45479)
0810 07:19:25.911066 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:25.911069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:25.911351 (+ 282us) server_negotiation.cc:407] Connection header received
0810 07:19:25.912011 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:25.912012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:25.912029 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:25.912086 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:25.913340 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.913903 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.915334 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.915603 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.915679 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:25.918304 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:25.918305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:25.918306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:25.918321 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:25.920278 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:25.920688 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:25.920690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:25.920690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:25.920788 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:25.921105 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:25.921106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:25.921107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:25.921290 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:25.921374 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:25.921682 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 07:19:25.921793 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:25.921970 15583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:25.910866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45479 (local address 127.14.164.2:47207)
0810 07:19:25.911114 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:25.911239 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:25.911880 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:25.912164 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:25.912169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:25.912978 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:25.913209 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:25.913215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.914031 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.914033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:25.915228 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:25.915231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:25.915705 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:25.915706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:25.915908 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:25.916507 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:25.916515 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:25.918180 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:25.920380 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:25.920387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:25.920389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:25.920620 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:25.920877 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:25.920878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:25.920879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:25.921016 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:25.921373 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:25.921379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:25.921520 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 07:19:25.921622 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:19:25.921778 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260810 07:19:25.922614 15581 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45479
I20260810 07:19:25.922745 15581 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:25.922963 15581 heartbeater.cc:511] Master 127.14.164.62:45479 requested a full tablet report, sending...
I20260810 07:19:25.923466 15074 ts_manager.cc:194] Registered new tserver with Master: 39688f27d8334debaa768ceba83f1ce9 (127.14.164.2:43159)
I20260810 07:19:25.924053 15074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:47207
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:25.947849 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45479
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:26.038354 15591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:26.038601 15591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:26.038645 15591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:26.039799 15591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:26.039870 15591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:26.039915 15591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:26.041488 15591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45479
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:26.041766 15591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:26.042014 15591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:26.047713 15598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:26.047758 15597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:26.047857 15591 server_base.cc:1034] running on GCE node
W20260810 07:19:26.047713 15601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:26.048175 15591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:26.048449 15591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:26.049605 15591 hybrid_clock.cc:648] HybridClock initialized: now 1786346366049592 us; error 28 us; skew 500 ppm
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:26.052269 15591 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:26.052870 15591 webserver.cc:460] Webserver started at http://127.14.164.3:40425/ using document root <none> and password file <none>
I20260810 07:19:26.053200 15591 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:26.053273 15591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:26.053421 15591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:26.054549 15591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "18c2b1a1561b4e31900aa56fbfec9b84"
format_stamp: "Formatted at 2026-08-10 07:19:26 on dist-test-slave-x5fp"
server_key: "4ec28e933fe074222b573a12a0480fc4"
server_key_iv: "f170aefb2faf72e14657bc94ba200a8b"
server_key_version: "encryptionkey@0"
I20260810 07:19:26.054955 15591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "18c2b1a1561b4e31900aa56fbfec9b84"
format_stamp: "Formatted at 2026-08-10 07:19:26 on dist-test-slave-x5fp"
server_key: "4ec28e933fe074222b573a12a0480fc4"
server_key_iv: "f170aefb2faf72e14657bc94ba200a8b"
server_key_version: "encryptionkey@0"
I20260810 07:19:26.058523 15591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:26.059437 15609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:26.059667 15591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:26.059782 15591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "18c2b1a1561b4e31900aa56fbfec9b84"
format_stamp: "Formatted at 2026-08-10 07:19:26 on dist-test-slave-x5fp"
server_key: "4ec28e933fe074222b573a12a0480fc4"
server_key_iv: "f170aefb2faf72e14657bc94ba200a8b"
server_key_version: "encryptionkey@0"
I20260810 07:19:26.059866 15591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:26.077616 15591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:26.080124 15591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:26.080299 15591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:26.080552 15591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:26.080942 15591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:26.081000 15591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:26.081045 15591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:26.081073 15591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:26.087740 15591 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:39373
I20260810 07:19:26.087790 15780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:39373 every 8 connection(s)
I20260810 07:19:26.088197 15591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:26.093842 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15591
I20260810 07:19:26.093950 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:26.094185 14992 external_mini_cluster.cc:1428] Setting key 64e8a4b915ca5e08017d10388a6225ee
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:26.099303 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:26.089141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:42553 (local address 127.14.164.62:45479)
0810 07:19:26.089226 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:26.089228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:26.089414 (+ 186us) server_negotiation.cc:407] Connection header received
0810 07:19:26.090063 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:26.090064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:26.090077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:26.090147 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:26.091236 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.091810 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.093061 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.093277 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.093347 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:26.096325 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:26.096326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:26.096327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:26.096342 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:26.097801 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:26.098248 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:26.098249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:26.098249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:26.098324 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:26.098593 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:26.098594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:26.098595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:26.098722 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:26.098801 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:26.099100 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 07:19:26.099199 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:19:26.099489 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:26.089060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45479 (local address 127.14.164.3:42553)
0810 07:19:26.089235 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:26.089285 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:26.089818 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:26.090214 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:26.090218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:26.090889 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:26.091113 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:26.091118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.091924 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.091925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:26.092952 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:26.092955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.093413 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.093415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:26.093555 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:26.094316 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:26.094327 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:26.096205 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:26.097902 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:26.097911 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:26.097914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:26.098163 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:26.098394 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:26.098395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:26.098396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:26.098520 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:26.098799 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:26.098807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:26.098955 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 07:19:26.099063 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:19:26.099234 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260810 07:19:26.100037 15781 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45479
I20260810 07:19:26.100131 15781 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:26.100359 15781 heartbeater.cc:511] Master 127.14.164.62:45479 requested a full tablet report, sending...
I20260810 07:19:26.100889 15074 ts_manager.cc:194] Registered new tserver with Master: 18c2b1a1561b4e31900aa56fbfec9b84 (127.14.164.3:39373)
I20260810 07:19:26.101338 15074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:42553
I20260810 07:19:26.107056 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:26.134174 15791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:26.125756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.14.164.1:42937)
0810 07:19:26.125970 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:26.125976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:26.125988 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:26.126016 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:26.126018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:26.126033 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:26.126114 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:26.126684 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.127425 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.128177 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.128531 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.128606 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:26.130924 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:26.130934 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:26.130935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:26.130942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:26.132690 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:26.133194 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:26.133198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:26.133199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:26.133281 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:26.133462 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:26.133463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:26.133463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:26.133610 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:26.133776 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:26.133819 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:19:26.133893 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
W20260810 07:19:26.134611 15240 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38022
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:26.159729 15791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:26.152410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.14.164.1:42937)
0810 07:19:26.152491 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:26.152492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:26.152501 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:26.152529 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:26.152530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:26.152541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:26.152603 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:26.153440 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.153939 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.154669 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:26.154875 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:26.154937 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:26.157469 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:26.157471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:26.157471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:26.157479 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:26.158755 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:26.159077 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:26.159078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:26.159078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:26.159131 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:26.159301 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:26.159301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:26.159302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:26.159407 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:26.159501 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:26.159564 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:19:26.159631 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:26.160356 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15166
I20260810 07:19:26.173120 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15384
I20260810 07:19:26.188517 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15591
I20260810 07:19:26.197441 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15020
2026-08-10T07:19:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2458 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15801](info): set up 2 sockets
Aug 10 07:19:26 dist-test-slave-x5fp krb5kdc[15801](info): commencing operation
krb5kdc: starting...
W20260810 07:19:27.475106 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:27 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:27.611618 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:41795
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:41795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:27.702122 15817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:27.702389 15817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:27.702415 15817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:27.703728 15817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:27.703786 15817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:27.703802 15817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:27.703815 15817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:27.703827 15817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:27.705566 15817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:41795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:41795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:27.705924 15817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:27.706192 15817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:27.712241 15827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:27.712271 15817 server_base.cc:1034] running on GCE node
W20260810 07:19:27.712247 15824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:27.712247 15825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:27.712944 15817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:27.713289 15817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:27.714499 15817 hybrid_clock.cc:648] HybridClock initialized: now 1786346367714477 us; error 39 us; skew 500 ppm
Aug 10 07:19:27 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:27.717450 15817 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:27.718046 15817 webserver.cc:460] Webserver started at http://127.14.164.62:41847/ using document root <none> and password file <none>
I20260810 07:19:27.718246 15817 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:27.718288 15817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:27.718393 15817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:27.719530 15817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "0384725e496a4243905ea0ffe7d1e7f8"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-x5fp"
server_key: "da437b2d45734545ea2b55874203b179"
server_key_iv: "e442cdb5a5684b5e651e8320fc0c1e07"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.719854 15817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "0384725e496a4243905ea0ffe7d1e7f8"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-x5fp"
server_key: "da437b2d45734545ea2b55874203b179"
server_key_iv: "e442cdb5a5684b5e651e8320fc0c1e07"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.721423 15817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:27.722390 15837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:27.722631 15817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:27.722735 15817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "0384725e496a4243905ea0ffe7d1e7f8"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-x5fp"
server_key: "da437b2d45734545ea2b55874203b179"
server_key_iv: "e442cdb5a5684b5e651e8320fc0c1e07"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.722801 15817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:27.731271 15817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:27.733713 15817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:27.733831 15817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:27.737882 15817 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:41795
I20260810 07:19:27.737989 15918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:41795 every 8 connection(s)
I20260810 07:19:27.738325 15817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:27.739377 15921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:27.741899 15921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8: Bootstrap starting.
I20260810 07:19:27.742523 15921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:27.742828 15921 log.cc:824] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:27.743763 15921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8: No bootstrap required, opened a new log
I20260810 07:19:27.745586 15921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0384725e496a4243905ea0ffe7d1e7f8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41795 } }
I20260810 07:19:27.745782 15921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:27.745839 15921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0384725e496a4243905ea0ffe7d1e7f8, State: Initialized, Role: FOLLOWER
I20260810 07:19:27.746009 15921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0384725e496a4243905ea0ffe7d1e7f8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41795 } }
I20260810 07:19:27.746104 15921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:27.746155 15921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:27.746210 15921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:27.746825 15921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0384725e496a4243905ea0ffe7d1e7f8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41795 } }
I20260810 07:19:27.746971 15921 leader_election.cc:302] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0384725e496a4243905ea0ffe7d1e7f8; no voters:
I20260810 07:19:27.747191 15921 leader_election.cc:288] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:27.747260 15926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:27.747475 15921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:27.747555 15926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [term 1 LEADER]: Becoming Leader. State: Replica: 0384725e496a4243905ea0ffe7d1e7f8, State: Running, Role: LEADER
I20260810 07:19:27.747689 15926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0384725e496a4243905ea0ffe7d1e7f8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41795 } }
I20260810 07:19:27.748194 15927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0384725e496a4243905ea0ffe7d1e7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0384725e496a4243905ea0ffe7d1e7f8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41795 } } }
I20260810 07:19:27.748328 15927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:27.748337 15928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0384725e496a4243905ea0ffe7d1e7f8. Latest consensus state: current_term: 1 leader_uuid: "0384725e496a4243905ea0ffe7d1e7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0384725e496a4243905ea0ffe7d1e7f8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41795 } } }
I20260810 07:19:27.748461 15928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:27.748771 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15817
I20260810 07:19:27.748881 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:27.748942 15939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:27.749078 14992 external_mini_cluster.cc:1428] Setting key f06951076f596f6fc0017fad68299b53
I20260810 07:19:27.749539 15939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:27.751468 15939 catalog_manager.cc:1347] Generated new cluster ID: ea82568498b04b0e96a9a99ef44b3139
I20260810 07:19:27.751530 15939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:19:27 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:27.760537 15939 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:27.761425 15939 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:27.762122 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:27.749687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.14.164.62:41795)
0810 07:19:27.750084 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:27.750094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:27.750129 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:19:27.750255 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:27.750263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:27.750341 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:27.750488 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:27.751321 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.752816 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.754011 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.754626 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.754745 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:27.757940 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:27.757945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:27.757945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:27.757960 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:27.760226 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.760691 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.760694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.760695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.760841 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.761078 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.761079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.761080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.761368 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:27.761568 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:27.761648 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:19:27.761933 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":347,"thread_start_us":221,"threads_started":1}
I20260810 07:19:27.765202 15939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0384725e496a4243905ea0ffe7d1e7f8: Generated new TSK 0
I20260810 07:19:27.765403 15939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:27.799463 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41795
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:27.895725 15969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:27.895975 15969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:27.896016 15969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:27.897303 15969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:27.897372 15969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:27.897416 15969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:27.898984 15969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:27.899264 15969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:27.899511 15969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:27.905730 15978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:27.905861 15969 server_base.cc:1034] running on GCE node
W20260810 07:19:27.905730 15981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:27.905977 15977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:27.906275 15969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:27.906585 15969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:27.907763 15969 hybrid_clock.cc:648] HybridClock initialized: now 1786346367907735 us; error 45 us; skew 500 ppm
Aug 10 07:19:27 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:27.911346 15969 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:27.911947 15969 webserver.cc:460] Webserver started at http://127.14.164.1:40641/ using document root <none> and password file <none>
I20260810 07:19:27.912277 15969 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:27.912354 15969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:27.912528 15969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:27.913815 15969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "3d073ca519b24567acf62c5a4fb4a16e"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-x5fp"
server_key: "c6fbb0a496e9711024b982dcfffce508"
server_key_iv: "b30f45ca16583a463f8b4254c7cb97f8"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.914224 15969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "3d073ca519b24567acf62c5a4fb4a16e"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-x5fp"
server_key: "c6fbb0a496e9711024b982dcfffce508"
server_key_iv: "b30f45ca16583a463f8b4254c7cb97f8"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.915870 15969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:27.916785 15994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:27.916987 15969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:27.917084 15969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "3d073ca519b24567acf62c5a4fb4a16e"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-x5fp"
server_key: "c6fbb0a496e9711024b982dcfffce508"
server_key_iv: "b30f45ca16583a463f8b4254c7cb97f8"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.917158 15969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:27.929828 15969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:27.932300 15969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:27.932495 15969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:27.932824 15969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:27.933183 15969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:27.933218 15969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:27.933274 15969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:27.933303 15969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:27.939519 15969 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:44759
I20260810 07:19:27.939568 16171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:44759 every 8 connection(s)
I20260810 07:19:27.939913 15969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:27.946375 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15969
I20260810 07:19:27.946463 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:27.946645 14992 external_mini_cluster.cc:1428] Setting key ecd19a8ebcc35b3a0e93a8f6d5d6cf22
Aug 10 07:19:27 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:27.951236 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:27.940672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37661 (local address 127.14.164.62:41795)
0810 07:19:27.940794 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:27.940797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:27.941011 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:19:27.942066 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:27.942067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:27.942082 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:27.942134 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:27.942838 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.943380 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.944418 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.944656 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.944757 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:27.947504 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:27.947505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:27.947506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:27.947515 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:27.949515 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.950054 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.950055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.950055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.950116 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.950465 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.950466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.950466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.950601 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:27.950670 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:27.951036 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 07:19:27.951141 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 07:19:27.951442 16175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:27.940660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41795 (local address 127.14.164.1:37661)
0810 07:19:27.940860 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:27.940901 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:27.941940 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:27.942210 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:27.942213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:27.942483 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:27.942696 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:27.942701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.943492 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.943493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:27.944323 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:27.944326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.944775 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.944776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:27.944940 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:27.945567 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:27.945576 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:27.947405 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:27.949587 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:27.949597 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:27.949601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:27.949961 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:27.950184 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:27.950185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:27.950185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:27.950356 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:27.950679 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:27.950687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:27.950878 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 07:19:27.951002 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:19:27.951178 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260810 07:19:27.952030 16172 heartbeater.cc:347] Connected to a master server at 127.14.164.62:41795
I20260810 07:19:27.952144 16172 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:27.952376 16172 heartbeater.cc:511] Master 127.14.164.62:41795 requested a full tablet report, sending...
I20260810 07:19:27.953070 15863 ts_manager.cc:194] Registered new tserver with Master: 3d073ca519b24567acf62c5a4fb4a16e (127.14.164.1:44759)
I20260810 07:19:27.953715 15863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:37661
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:27.981882 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41795
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:28.071738 16181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:28.071966 16181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:28.071993 16181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:28.073176 16181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:28.073222 16181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:28.073251 16181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:28.074762 16181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:28.074976 16181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:28.075181 16181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:28.080605 16192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:28.080683 16189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:28.080899 16186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:28.081254 16181 server_base.cc:1034] running on GCE node
I20260810 07:19:28.081488 16181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:28.081728 16181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:28.082966 16181 hybrid_clock.cc:648] HybridClock initialized: now 1786346368082938 us; error 32 us; skew 500 ppm
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:28.086236 16181 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:28.086863 16181 webserver.cc:460] Webserver started at http://127.14.164.2:38923/ using document root <none> and password file <none>
I20260810 07:19:28.087131 16181 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:28.087205 16181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:28.087364 16181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:28.088521 16181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "7cbe227f5bce4d7f92b08308731ad3ed"
format_stamp: "Formatted at 2026-08-10 07:19:28 on dist-test-slave-x5fp"
server_key: "b848a9f758f638b1bcbe3ddbe97e95c0"
server_key_iv: "4197392206125cb12bf337ab1dbec1af"
server_key_version: "encryptionkey@0"
I20260810 07:19:28.088943 16181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "7cbe227f5bce4d7f92b08308731ad3ed"
format_stamp: "Formatted at 2026-08-10 07:19:28 on dist-test-slave-x5fp"
server_key: "b848a9f758f638b1bcbe3ddbe97e95c0"
server_key_iv: "4197392206125cb12bf337ab1dbec1af"
server_key_version: "encryptionkey@0"
I20260810 07:19:28.090417 16181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:28.091313 16202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:28.091558 16181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:28.091681 16181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "7cbe227f5bce4d7f92b08308731ad3ed"
format_stamp: "Formatted at 2026-08-10 07:19:28 on dist-test-slave-x5fp"
server_key: "b848a9f758f638b1bcbe3ddbe97e95c0"
server_key_iv: "4197392206125cb12bf337ab1dbec1af"
server_key_version: "encryptionkey@0"
I20260810 07:19:28.091763 16181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:28.102160 16181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:28.104463 16181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:28.104640 16181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:28.104934 16181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:28.105268 16181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:28.105324 16181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:28.105370 16181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:28.105393 16181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:28.111771 16181 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:40061
I20260810 07:19:28.111867 16398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:40061 every 8 connection(s)
I20260810 07:19:28.112279 16181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:28.118669 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16181
I20260810 07:19:28.118788 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:28.118990 14992 external_mini_cluster.cc:1428] Setting key 926283dd72dc129b969417f1c354bfea
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:28.124027 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.113156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:59847 (local address 127.14.164.62:41795)
0810 07:19:28.113253 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.113256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.113488 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:19:28.114437 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.114439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.114451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.114514 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:28.115147 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.115714 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.116962 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.117174 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.117257 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.120022 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:28.120024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:28.120024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:28.120033 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:28.122049 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.122673 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.122674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.122675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.122745 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.123139 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.123140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.123140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.123282 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:28.123350 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.123775 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.123880 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:19:28.124056 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.113124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41795 (local address 127.14.164.2:59847)
0810 07:19:28.113343 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.113381 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.114312 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.114569 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.114573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.114838 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:28.115049 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.115053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.115838 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.115839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.116790 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.116794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.117292 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.117293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.117437 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.118013 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:28.118022 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:28.119909 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:28.122156 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:28.122166 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:28.122169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:28.122564 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:28.122869 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:28.122870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:28.122870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:28.123037 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:28.123368 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:28.123376 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:28.123563 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.123720 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.123874 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":168,"threads_started":1}
I20260810 07:19:28.126093 16402 heartbeater.cc:347] Connected to a master server at 127.14.164.62:41795
I20260810 07:19:28.126235 16402 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:28.126477 16402 heartbeater.cc:511] Master 127.14.164.62:41795 requested a full tablet report, sending...
I20260810 07:19:28.126998 15863 ts_manager.cc:194] Registered new tserver with Master: 7cbe227f5bce4d7f92b08308731ad3ed (127.14.164.2:40061)
I20260810 07:19:28.127444 15863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:59847
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:28.153419 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41795
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:28.242673 16412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:28.242894 16412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:28.242920 16412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:28.244208 16412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:28.244256 16412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:28.244290 16412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:28.245864 16412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:28.246122 16412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:28.246378 16412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:28.252120 16412 server_base.cc:1034] running on GCE node
W20260810 07:19:28.252154 16423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:28.252202 16426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:28.252175 16424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:28.252535 16412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:28.252789 16412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:28.253973 16412 hybrid_clock.cc:648] HybridClock initialized: now 1786346368253960 us; error 30 us; skew 500 ppm
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:28.256523 16412 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:28.257020 16412 webserver.cc:460] Webserver started at http://127.14.164.3:40227/ using document root <none> and password file <none>
I20260810 07:19:28.257263 16412 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:28.257306 16412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:28.257395 16412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:28.258373 16412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "edd8d5df8f1a4060b2eb89151d9212ae"
format_stamp: "Formatted at 2026-08-10 07:19:28 on dist-test-slave-x5fp"
server_key: "64478af83fb9dd8269c5325b70f10020"
server_key_iv: "f5c58317aad4aa08ba57cb2a0253c786"
server_key_version: "encryptionkey@0"
I20260810 07:19:28.258678 16412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "edd8d5df8f1a4060b2eb89151d9212ae"
format_stamp: "Formatted at 2026-08-10 07:19:28 on dist-test-slave-x5fp"
server_key: "64478af83fb9dd8269c5325b70f10020"
server_key_iv: "f5c58317aad4aa08ba57cb2a0253c786"
server_key_version: "encryptionkey@0"
I20260810 07:19:28.259976 16412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:19:28.260646 16438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:28.260879 16412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:28.260988 16412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "edd8d5df8f1a4060b2eb89151d9212ae"
format_stamp: "Formatted at 2026-08-10 07:19:28 on dist-test-slave-x5fp"
server_key: "64478af83fb9dd8269c5325b70f10020"
server_key_iv: "f5c58317aad4aa08ba57cb2a0253c786"
server_key_version: "encryptionkey@0"
I20260810 07:19:28.261068 16412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:28.270977 16412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:28.273346 16412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:28.273512 16412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:28.273753 16412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:28.274078 16412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:28.274114 16412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:28.274137 16412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:28.274179 16412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:28.280054 16412 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:45705
I20260810 07:19:28.280200 16615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:45705 every 8 connection(s)
I20260810 07:19:28.280475 16412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:28.290180 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16412
I20260810 07:19:28.290310 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:28.290475 14992 external_mini_cluster.cc:1428] Setting key 4e6da0d21593f7a843ef18715adb2a0a
I20260810 07:19:28.290512 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.281343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:51763 (local address 127.14.164.62:41795)
0810 07:19:28.281393 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.281395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.281617 (+ 222us) server_negotiation.cc:407] Connection header received
0810 07:19:28.282550 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.282551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.282572 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.282635 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:28.283251 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.283736 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.284816 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.285013 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.285086 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.287502 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:28.287503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:28.287503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:28.287511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:28.289107 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.289523 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.289524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.289524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.289581 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.289808 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.289808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.289808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.289917 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:28.289969 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.290311 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.290408 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:28.290767 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.281293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41795 (local address 127.14.164.3:51763)
0810 07:19:28.281482 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.281525 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.282458 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.282714 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.282717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.282983 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:28.283176 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.283180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.283823 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.283824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.284633 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.284635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.285166 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.285167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.285280 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.285814 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:28.285824 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:28.287337 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:28.289187 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:28.289194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:28.289204 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:28.289453 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:28.289642 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:28.289643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:28.289643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:28.289752 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:28.289977 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:28.289982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:28.290135 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.290275 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.290481 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260810 07:19:28.291325 16616 heartbeater.cc:347] Connected to a master server at 127.14.164.62:41795
I20260810 07:19:28.291460 16616 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:28.291666 16616 heartbeater.cc:511] Master 127.14.164.62:41795 requested a full tablet report, sending...
I20260810 07:19:28.292171 15863 ts_manager.cc:194] Registered new tserver with Master: edd8d5df8f1a4060b2eb89151d9212ae (127.14.164.3:45705)
I20260810 07:19:28.292604 15863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:51763
I20260810 07:19:28.302822 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:28.310937 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.304547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.14.164.62:41795)
0810 07:19:28.304609 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.304611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.304662 (+ 51us) server_negotiation.cc:407] Connection header received
0810 07:19:28.304706 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.304707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.304808 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.304886 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:28.305432 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.305925 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.306659 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.306861 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.306950 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.308164 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:28.308166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:28.308166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:28.308174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:28.309720 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.310113 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.310114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.310114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.310175 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.310450 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.310450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.310451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.310604 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:28.310685 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.310795 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.310869 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:19:28.313462 15863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:28.315114 15863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:28.322660 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.317925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45705 (local address 127.0.0.1:46082)
0810 07:19:28.318358 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.318368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.318404 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.318617 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.318621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.318623 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:28.318705 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.318707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.319924 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.319925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.321560 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.321562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.322404 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.322405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.322457 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.322460 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.322492 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.322556 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260810 07:19:28.322665 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.318447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.14.164.1:44759)
0810 07:19:28.318875 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.318880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.318890 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:28.318931 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.318933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.318949 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.319044 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:28.319204 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.320290 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.321318 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.322124 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.322206 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.322370 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.322541 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.322593 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":114,"threads_started":1}
I20260810 07:19:28.322824 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.317922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:40061 (local address 127.0.0.1:52046)
0810 07:19:28.318042 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.318085 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.318175 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.318676 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.318677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.318679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:28.318714 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.318715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.320233 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.320234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.321326 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.321329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.322596 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.322596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.322653 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.322655 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.322709 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.322752 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
I20260810 07:19:28.322826 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.317925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44759 (local address 127.0.0.1:52578)
0810 07:19:28.318205 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.318219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.318276 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.319053 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.319053 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.319056 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:28.319114 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.319117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.320397 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.320398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.321214 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.321217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.322297 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.322299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.322420 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.322440 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.322518 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.322555 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260810 07:19:28.323268 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.318214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.14.164.3:45705)
0810 07:19:28.318392 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.318397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.318409 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:28.318482 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.318484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.318502 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.318590 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:28.318850 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.319815 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.321643 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.322294 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.322978 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.323076 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.323106 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.323157 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260810 07:19:28.323608 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.318255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.14.164.2:40061)
0810 07:19:28.318489 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.318499 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.318508 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:28.318542 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.318543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.318574 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.318796 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:28.318932 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.320102 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.321433 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.322120 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.322206 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.322392 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.323478 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.323531 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":85,"threads_started":1}
I20260810 07:19:28.323752 16059 tablet_service.cc:1467] Processing CreateTablet for tablet 15bfcf2814e74069aa3e7a1bc173a3f2 (DEFAULT_TABLE table=test-table [id=bea4b0c20f5547efa52d25dcb657746f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:28.324165 16059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bfcf2814e74069aa3e7a1bc173a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:28.324128 16520 tablet_service.cc:1467] Processing CreateTablet for tablet 15bfcf2814e74069aa3e7a1bc173a3f2 (DEFAULT_TABLE table=test-table [id=bea4b0c20f5547efa52d25dcb657746f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:28.324231 16283 tablet_service.cc:1467] Processing CreateTablet for tablet 15bfcf2814e74069aa3e7a1bc173a3f2 (DEFAULT_TABLE table=test-table [id=bea4b0c20f5547efa52d25dcb657746f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:28.324483 16520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bfcf2814e74069aa3e7a1bc173a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:28.324548 16283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bfcf2814e74069aa3e7a1bc173a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:28.327394 16638 tablet_bootstrap.cc:492] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: Bootstrap starting.
I20260810 07:19:28.327936 16639 tablet_bootstrap.cc:492] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: Bootstrap starting.
I20260810 07:19:28.328188 16640 tablet_bootstrap.cc:492] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: Bootstrap starting.
I20260810 07:19:28.328291 16638 tablet_bootstrap.cc:654] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:28.328624 16639 tablet_bootstrap.cc:654] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:28.328857 16640 tablet_bootstrap.cc:654] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:28.328962 16638 log.cc:824] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:28.329241 16639 log.cc:824] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:28.329481 16640 log.cc:824] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:28.330087 16639 tablet_bootstrap.cc:492] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: No bootstrap required, opened a new log
I20260810 07:19:28.330173 16639 ts_tablet_manager.cc:1397] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:28.330331 16640 tablet_bootstrap.cc:492] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: No bootstrap required, opened a new log
I20260810 07:19:28.330392 16640 ts_tablet_manager.cc:1397] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:28.331650 16639 raft_consensus.cc:348] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.331830 16639 raft_consensus.cc:374] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:28.331881 16639 raft_consensus.cc:729] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cbe227f5bce4d7f92b08308731ad3ed, State: Initialized, Role: FOLLOWER
I20260810 07:19:28.331810 16640 raft_consensus.cc:348] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.332018 16640 raft_consensus.cc:374] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:28.332001 16639 consensus_queue.cc:260] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.332082 16640 raft_consensus.cc:729] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edd8d5df8f1a4060b2eb89151d9212ae, State: Initialized, Role: FOLLOWER
I20260810 07:19:28.332209 16640 consensus_queue.cc:260] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.332278 16639 ts_tablet_manager.cc:1428] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:28.332334 16402 heartbeater.cc:503] Master 127.14.164.62:41795 was elected leader, sending a full tablet report...
I20260810 07:19:28.332541 16640 ts_tablet_manager.cc:1428] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:28.332661 16616 heartbeater.cc:503] Master 127.14.164.62:41795 was elected leader, sending a full tablet report...
I20260810 07:19:28.332721 16638 tablet_bootstrap.cc:492] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: No bootstrap required, opened a new log
I20260810 07:19:28.332829 16638 ts_tablet_manager.cc:1397] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 07:19:28.334589 16638 raft_consensus.cc:348] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.334764 16638 raft_consensus.cc:374] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:28.334816 16638 raft_consensus.cc:729] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d073ca519b24567acf62c5a4fb4a16e, State: Initialized, Role: FOLLOWER
I20260810 07:19:28.334933 16638 consensus_queue.cc:260] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.335244 16638 ts_tablet_manager.cc:1428] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:28.335322 16172 heartbeater.cc:503] Master 127.14.164.62:41795 was elected leader, sending a full tablet report...
W20260810 07:19:28.362931 16404 tablet.cc:2369] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:19:28.440702 16173 tablet.cc:2369] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:28.441417 16644 raft_consensus.cc:482] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:28.441526 16644 raft_consensus.cc:504] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.441900 16644 leader_election.cc:288] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d073ca519b24567acf62c5a4fb4a16e (127.14.164.1:44759), edd8d5df8f1a4060b2eb89151d9212ae (127.14.164.3:45705)
I20260810 07:19:28.446121 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.442212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:39937 (local address 127.14.164.1:44759)
0810 07:19:28.442257 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.442259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.442464 (+ 205us) server_negotiation.cc:407] Connection header received
0810 07:19:28.442492 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.442493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.442502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.442562 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:28.442706 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.443394 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.445056 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.445632 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.445717 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.445727 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.445999 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.446043 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:28.446121 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.442177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:54699 (local address 127.14.164.3:45705)
0810 07:19:28.442235 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.442238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.442251 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:28.442309 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.442310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.442329 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.442384 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:28.442591 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.443389 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.445056 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.445639 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.445717 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.445727 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.445999 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.446044 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:19:28.446148 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.442084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45705 (local address 127.14.164.2:54699)
0810 07:19:28.442152 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.442166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.442227 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.442414 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.442415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.442417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:28.442500 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.442502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.443502 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.443503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.444951 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.444955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.445792 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.445794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.445898 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.445901 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.445978 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.446033 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":98}
I20260810 07:19:28.446156 16647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.442104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44759 (local address 127.14.164.2:39937)
0810 07:19:28.442373 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:28.442383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:28.442436 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:28.442579 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:28.442580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:28.442581 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:28.442640 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.442641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.443502 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.443503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.444951 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:28.444955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.445792 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.445794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:28.445895 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.445901 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:28.445978 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:19:28.446033 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":162,"thread_start_us":138,"threads_started":1}
I20260810 07:19:28.446645 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bfcf2814e74069aa3e7a1bc173a3f2" candidate_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" is_pre_election: true
I20260810 07:19:28.446659 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bfcf2814e74069aa3e7a1bc173a3f2" candidate_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d073ca519b24567acf62c5a4fb4a16e" is_pre_election: true
I20260810 07:19:28.446846 16552 raft_consensus.cc:2454] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cbe227f5bce4d7f92b08308731ad3ed in term 0.
I20260810 07:19:28.446846 16090 raft_consensus.cc:2454] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cbe227f5bce4d7f92b08308731ad3ed in term 0.
I20260810 07:19:28.447151 16208 leader_election.cc:302] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cbe227f5bce4d7f92b08308731ad3ed, edd8d5df8f1a4060b2eb89151d9212ae; no voters:
I20260810 07:19:28.447342 16644 raft_consensus.cc:2781] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:28.447438 16644 raft_consensus.cc:482] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:28.447468 16644 raft_consensus.cc:3037] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:28.448326 16644 raft_consensus.cc:504] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.448491 16644 leader_election.cc:288] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [CANDIDATE]: Term 1 election: Requested vote from peers 3d073ca519b24567acf62c5a4fb4a16e (127.14.164.1:44759), edd8d5df8f1a4060b2eb89151d9212ae (127.14.164.3:45705)
I20260810 07:19:28.448691 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bfcf2814e74069aa3e7a1bc173a3f2" candidate_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d073ca519b24567acf62c5a4fb4a16e"
I20260810 07:19:28.448652 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bfcf2814e74069aa3e7a1bc173a3f2" candidate_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edd8d5df8f1a4060b2eb89151d9212ae"
I20260810 07:19:28.448844 16552 raft_consensus.cc:3037] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:28.448848 16090 raft_consensus.cc:3037] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:28.449741 16552 raft_consensus.cc:2454] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cbe227f5bce4d7f92b08308731ad3ed in term 1.
I20260810 07:19:28.449879 16090 raft_consensus.cc:2454] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 3d073ca519b24567acf62c5a4fb4a16e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cbe227f5bce4d7f92b08308731ad3ed in term 1.
I20260810 07:19:28.449946 16208 leader_election.cc:302] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cbe227f5bce4d7f92b08308731ad3ed, edd8d5df8f1a4060b2eb89151d9212ae; no voters:
I20260810 07:19:28.450052 16644 raft_consensus.cc:2781] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:28.450292 16644 raft_consensus.cc:686] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [term 1 LEADER]: Becoming Leader. State: Replica: 7cbe227f5bce4d7f92b08308731ad3ed, State: Running, Role: LEADER
I20260810 07:19:28.450431 16644 consensus_queue.cc:237] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } }
I20260810 07:19:28.451397 15861 catalog_manager.cc:5495] T 15bfcf2814e74069aa3e7a1bc173a3f2 P 7cbe227f5bce4d7f92b08308731ad3ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cbe227f5bce4d7f92b08308731ad3ed (127.14.164.2). New cstate: current_term: 1 leader_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbe227f5bce4d7f92b08308731ad3ed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d073ca519b24567acf62c5a4fb4a16e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edd8d5df8f1a4060b2eb89151d9212ae" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45705 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:28.472407 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.464106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.14.164.1:44759)
0810 07:19:28.464227 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.464229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.464320 (+ 91us) server_negotiation.cc:407] Connection header received
0810 07:19:28.464419 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.464420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.464433 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.464493 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:28.465195 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.465685 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.466425 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.466628 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.466709 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.469316 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:28.469331 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:28.469332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:28.469340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:28.471157 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.471514 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.471517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.471517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.471584 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.471838 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.471839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.471840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.472010 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:28.472143 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.472257 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.472331 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:28.498414 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.490742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.14.164.1:44759)
0810 07:19:28.490818 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.490820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.490911 (+ 91us) server_negotiation.cc:407] Connection header received
0810 07:19:28.490968 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.490968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.490978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.491028 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:28.491591 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.492078 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.492920 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.493120 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.493186 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.495603 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:28.495605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:28.495605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:28.495613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:28.497174 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.497518 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.497518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.497519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.497578 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.497779 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.497780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.497780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.497878 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:28.497953 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.498157 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.498275 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[15801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:28.523882 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:28.516395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.14.164.1:44759)
0810 07:19:28.516474 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:28.516476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:28.516517 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:28.516628 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:28.516630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:28.516641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:28.516700 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:28.517366 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.517855 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.518573 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:28.518750 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:28.518832 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:28.521342 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:28.521344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:28.521344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:28.521352 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:28.522759 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.523168 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.523170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.523170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.523256 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:28.523443 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:28.523444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:28.523444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:28.523569 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:28.523661 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:28.523749 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 07:19:28.523813 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 07:19:28.524406 16029 tablet_service.cc:2908] Rejecting scan request for tablet 15bfcf2814e74069aa3e7a1bc173a3f2: Uninitialized: clean time has not yet been initialized
I20260810 07:19:28.525358 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15969
W20260810 07:19:28.532023 16617 tablet.cc:2369] T 15bfcf2814e74069aa3e7a1bc173a3f2 P edd8d5df8f1a4060b2eb89151d9212ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:28.538205 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16181
I20260810 07:19:28.554759 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16412
I20260810 07:19:28.566289 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15817
2026-08-10T07:19:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2369 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[16671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[16671](info): set up 2 sockets
Aug 10 07:19:28 dist-test-slave-x5fp krb5kdc[16671](info): commencing operation
krb5kdc: starting...
W20260810 07:19:29.831817 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:29 dist-test-slave-x5fp krb5kdc[16671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:29.965668 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:44631
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:44631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:30.052101 16695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:30.052325 16695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:30.052349 16695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:30.053579 16695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:30.053624 16695 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:19:30.053638 16695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:30.053651 16695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:30.053664 16695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:30.053676 16695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:30.055233 16695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:44631
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:44631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:30.055495 16695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:30.055761 16695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:30.061233 16708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.061268 16704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.061321 16702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:30.061533 16695 server_base.cc:1034] running on GCE node
I20260810 07:19:30.061815 16695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:30.062136 16695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:30.063344 16695 hybrid_clock.cc:648] HybridClock initialized: now 1786346370063334 us; error 41 us; skew 500 ppm
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:30.066308 16695 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:30.066823 16695 webserver.cc:460] Webserver started at http://127.14.164.62:37049/ using document root <none> and password file <none>
I20260810 07:19:30.067034 16695 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:30.067080 16695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:30.067178 16695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:30.068324 16695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "a594d9319e80454da14e430fe2b30d40"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "4796745e5b4da53cf6b43eeaaa211916"
server_key_iv: "851ced04ef5b6a8ec95a09d0f52a8d58"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.068658 16695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "a594d9319e80454da14e430fe2b30d40"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "4796745e5b4da53cf6b43eeaaa211916"
server_key_iv: "851ced04ef5b6a8ec95a09d0f52a8d58"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.070182 16695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:30.071125 16722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.071426 16695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:30.071535 16695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "a594d9319e80454da14e430fe2b30d40"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "4796745e5b4da53cf6b43eeaaa211916"
server_key_iv: "851ced04ef5b6a8ec95a09d0f52a8d58"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.071614 16695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:30.103503 16695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:30.106088 16695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:30.106243 16695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:30.111477 16695 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:44631
I20260810 07:19:30.111553 16819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:44631 every 8 connection(s)
I20260810 07:19:30.111927 16695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:30.112093 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16695
I20260810 07:19:30.112216 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:30.112457 14992 external_mini_cluster.cc:1428] Setting key 6dbc5e7471678f16dc9e14c0800b333c
I20260810 07:19:30.112912 16821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.115906 16821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: Bootstrap starting.
I20260810 07:19:30.116796 16821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.117110 16821 log.cc:824] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:30.117965 16821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: No bootstrap required, opened a new log
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:30.120553 16821 raft_consensus.cc:348] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a594d9319e80454da14e430fe2b30d40" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44631 } }
I20260810 07:19:30.120692 16821 raft_consensus.cc:374] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.120782 16821 raft_consensus.cc:729] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a594d9319e80454da14e430fe2b30d40, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.120918 16821 consensus_queue.cc:260] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a594d9319e80454da14e430fe2b30d40" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44631 } }
I20260810 07:19:30.121009 16821 raft_consensus.cc:388] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:30.121045 16821 raft_consensus.cc:482] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:30.121093 16821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.121784 16821 raft_consensus.cc:504] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a594d9319e80454da14e430fe2b30d40" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44631 } }
I20260810 07:19:30.121891 16821 leader_election.cc:302] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a594d9319e80454da14e430fe2b30d40; no voters:
I20260810 07:19:30.122100 16821 leader_election.cc:288] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:30.122148 16828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:30.122318 16828 raft_consensus.cc:686] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [term 1 LEADER]: Becoming Leader. State: Replica: a594d9319e80454da14e430fe2b30d40, State: Running, Role: LEADER
I20260810 07:19:30.122393 16821 sys_catalog.cc:564] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:30.122416 16828 consensus_queue.cc:237] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a594d9319e80454da14e430fe2b30d40" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44631 } }
I20260810 07:19:30.122753 16828 sys_catalog.cc:455] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a594d9319e80454da14e430fe2b30d40. Latest consensus state: current_term: 1 leader_uuid: "a594d9319e80454da14e430fe2b30d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a594d9319e80454da14e430fe2b30d40" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44631 } } }
I20260810 07:19:30.122764 16829 sys_catalog.cc:455] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a594d9319e80454da14e430fe2b30d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a594d9319e80454da14e430fe2b30d40" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44631 } } }
I20260810 07:19:30.122850 16828 sys_catalog.cc:458] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:30.122879 16829 sys_catalog.cc:458] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:30.123566 16836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:30.124204 16836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:30.125195 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.113026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60924 (local address 127.14.164.62:44631)
0810 07:19:30.113227 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.113237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.113271 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:19:30.114059 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.114068 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.114123 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.114322 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.115031 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.116052 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.117197 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.117800 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.117893 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.120621 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.120625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.120626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.120644 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.122846 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.123339 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.123343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.123343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.123455 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.123683 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.123684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.123684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.124034 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.124215 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.124857 (+ 642us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.125001 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260810 07:19:30.125907 16836 catalog_manager.cc:1347] Generated new cluster ID: 8aa9f9e137e94668914de631dd7bbbc1
I20260810 07:19:30.125970 16836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:30.141424 16836 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:30.142030 16836 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:30.145866 16836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: Generated new TSK 0
I20260810 07:19:30.146126 16836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:30.176111 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44631
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:30.263029 16861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:30.263247 16861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:30.263274 16861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:30.264401 16861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:30.264437 16861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:30.264458 16861 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:30.264477 16861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:30.266026 16861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:44631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:30.266239 16861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:30.266434 16861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:30.271862 16868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.271891 16871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.271927 16867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:30.272044 16861 server_base.cc:1034] running on GCE node
I20260810 07:19:30.272385 16861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:30.272691 16861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:30.273923 16861 hybrid_clock.cc:648] HybridClock initialized: now 1786346370273878 us; error 39 us; skew 500 ppm
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:30.276858 16861 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:30.277297 16861 webserver.cc:460] Webserver started at http://127.14.164.1:37617/ using document root <none> and password file <none>
I20260810 07:19:30.277515 16861 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:30.277558 16861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:30.277652 16861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:30.278700 16861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "16d36ea39c2d42468f741ad44179cbd5"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "ecc5c4b087b372beed7c2f29a10820d5"
server_key_iv: "1607381e8948941fb6ceec806aaee35e"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.279016 16861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "16d36ea39c2d42468f741ad44179cbd5"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "ecc5c4b087b372beed7c2f29a10820d5"
server_key_iv: "1607381e8948941fb6ceec806aaee35e"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.280318 16861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:30.281183 16885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.281361 16861 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:30.281458 16861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "16d36ea39c2d42468f741ad44179cbd5"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "ecc5c4b087b372beed7c2f29a10820d5"
server_key_iv: "1607381e8948941fb6ceec806aaee35e"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.281517 16861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:30.295017 16861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:30.297459 16861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:30.297577 16861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:30.298067 16861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:30.298112 16861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.298138 16861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:30.298151 16861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.305356 16861 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:42987
I20260810 07:19:30.305426 17070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:42987 every 8 connection(s)
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:30.305797 16861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:30.308957 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.298601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:35327 (local address 127.14.164.62:44631)
0810 07:19:30.298678 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.298680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.298967 (+ 287us) server_negotiation.cc:407] Connection header received
0810 07:19:30.299460 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.299462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.299475 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.299581 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.300486 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.301407 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.303064 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.303411 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.303499 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.306311 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.306313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.306313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.306320 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.307487 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.307966 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.307967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.307967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.308042 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.308315 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.308316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.308316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.308434 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.308490 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.308706 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.308881 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:19:30.309140 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.298579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.14.164.1:35327)
0810 07:19:30.298823 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.298866 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.299361 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.299649 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.299652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.300075 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.300375 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.300382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.301511 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.301512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.302963 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.302967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.303503 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.303504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.303613 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.304204 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.304221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.306217 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.307578 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.307586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.307588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.307874 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.308136 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.308136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.308137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.308253 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.308495 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.308500 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.308613 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.308696 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.308917 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":119,"threads_started":1}
I20260810 07:19:30.309659 17072 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44631
I20260810 07:19:30.309748 17072 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:30.309939 17072 heartbeater.cc:511] Master 127.14.164.62:44631 requested a full tablet report, sending...
I20260810 07:19:30.310498 16745 ts_manager.cc:194] Registered new tserver with Master: 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987)
I20260810 07:19:30.311261 16745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:35327
I20260810 07:19:30.312687 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16861
I20260810 07:19:30.312861 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:30.313054 14992 external_mini_cluster.cc:1428] Setting key c6efee9aad995894c75605038b220aff
I20260810 07:19:30.317216 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.310785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:51661 (local address 127.14.164.62:44631)
0810 07:19:30.310829 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.310831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.310960 (+ 129us) server_negotiation.cc:407] Connection header received
0810 07:19:30.310984 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.310985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.310994 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.311060 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.311358 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.312053 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.312985 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.313244 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.313320 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.314299 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.314301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.314301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.314308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.315862 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.316309 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.316310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.316310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.316369 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.316590 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.316591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.316591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.316694 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.316870 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.317088 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.317155 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:19:30.317325 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.310732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.14.164.1:51661)
0810 07:19:30.310883 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.310901 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.310950 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.311062 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.311063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.311234 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.311284 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.311285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.312156 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.312156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.312867 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.312870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.313338 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.313339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.313419 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.313924 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.313929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.314209 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.315953 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.315954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.315955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.316212 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.316449 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.316450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.316450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.316524 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.316914 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.316915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.316976 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.317063 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.317182 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:30.344053 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44631
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:30.441318 17092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:30.441601 17092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:30.441640 17092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:30.443887 17092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:30.443966 17092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:30.444006 17092 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:30.444034 17092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:30.446609 17092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:44631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:30.446959 17092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:30.447242 17092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:30.453487 17102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.453588 17106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.453504 17101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:30.453696 17092 server_base.cc:1034] running on GCE node
I20260810 07:19:30.453974 17092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:30.454280 17092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:30.455467 17092 hybrid_clock.cc:648] HybridClock initialized: now 1786346370455440 us; error 60 us; skew 500 ppm
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:30.458516 17092 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:30.459112 17092 webserver.cc:460] Webserver started at http://127.14.164.2:38805/ using document root <none> and password file <none>
I20260810 07:19:30.459600 17092 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:30.459683 17092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:30.459908 17092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:30.461604 17092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "d345929982afb5eb1f8b44efb5736a5e"
server_key_iv: "31529b811d0724e0a65ec76fcfd4946a"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.462049 17092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "d345929982afb5eb1f8b44efb5736a5e"
server_key_iv: "31529b811d0724e0a65ec76fcfd4946a"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.463949 17092 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:30.464995 17115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.465199 17092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:30.465317 17092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "d345929982afb5eb1f8b44efb5736a5e"
server_key_iv: "31529b811d0724e0a65ec76fcfd4946a"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.465410 17092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:30.478206 17092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:30.480608 17092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:30.480825 17092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:30.481433 17092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:30.481500 17092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.481566 17092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:30.481598 17092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:30.489429 17092 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:41435
I20260810 07:19:30.489576 17300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:41435 every 8 connection(s)
I20260810 07:19:30.489880 17092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:30.491837 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.482055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:41531 (local address 127.14.164.62:44631)
0810 07:19:30.482118 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.482120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.482372 (+ 252us) server_negotiation.cc:407] Connection header received
0810 07:19:30.482892 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.482894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.482909 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.482994 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.484002 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.484944 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.486040 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.486266 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.486348 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.488607 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.488609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.488609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.488620 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.490421 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.490834 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.490835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.490835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.490941 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.491256 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.491258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.491258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.491388 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.491445 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.491670 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.491770 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:19:30.491997 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.482036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.14.164.2:41531)
0810 07:19:30.482232 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.482273 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.482789 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.483153 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.483163 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.483557 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.483868 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.483875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.485050 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.485051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.485937 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.485940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.486386 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.486387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.486522 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.487097 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.487108 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.488525 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.490506 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.490509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.490511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.490737 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.491035 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.491036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.491036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.491153 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.491443 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.491446 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.491560 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.491638 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.491766 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
I20260810 07:19:30.492527 17301 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44631
I20260810 07:19:30.492621 17301 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:30.492849 17301 heartbeater.cc:511] Master 127.14.164.62:44631 requested a full tablet report, sending...
I20260810 07:19:30.493285 16745 ts_manager.cc:194] Registered new tserver with Master: e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.493623 16745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:41531
I20260810 07:19:30.493942 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17092
I20260810 07:19:30.494045 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:30.494248 14992 external_mini_cluster.cc:1428] Setting key f96fb8b3a8859fc135a16ec59f594074
I20260810 07:19:30.501011 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.493740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.14.164.2:35835)
0810 07:19:30.494461 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.494472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.494554 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.494715 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.494716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.494950 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.495025 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.495027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.495657 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.495658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.496521 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.496524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.497052 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.497053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.497138 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.497658 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.497663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.497969 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.499815 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.499816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.499817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.499994 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.500222 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.500223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.500224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.500300 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.500676 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.500677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.500758 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.500850 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.500937 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":73,"threads_started":1}
I20260810 07:19:30.501106 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.493912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:35835 (local address 127.14.164.62:44631)
0810 07:19:30.493958 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.493960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.494541 (+ 581us) server_negotiation.cc:407] Connection header received
0810 07:19:30.494628 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.494629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.494640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.494699 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.495090 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.495553 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.496624 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.496943 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.497049 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.498066 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.498067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.498068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.498075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.499705 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.500068 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.500069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.500070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.500139 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.500370 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.500371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.500372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.500606 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.500683 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.500906 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.501008 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:30.524868 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44631
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:30.611428 17315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:30.611644 17315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:30.611667 17315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:30.612874 17315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:30.612915 17315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:30.612936 17315 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:30.612954 17315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:30.614373 17315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:44631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:30.614573 17315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:30.614753 17315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:30.620265 17315 server_base.cc:1034] running on GCE node
W20260810 07:19:30.620229 17323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.620213 17329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.620231 17324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:30.620676 17315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:30.621030 17315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:30.622262 17315 hybrid_clock.cc:648] HybridClock initialized: now 1786346370622242 us; error 39 us; skew 500 ppm
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:30.625487 17315 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:30.626010 17315 webserver.cc:460] Webserver started at http://127.14.164.3:37359/ using document root <none> and password file <none>
I20260810 07:19:30.626284 17315 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:30.626338 17315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:30.626525 17315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:30.627667 17315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "26f9ea9b3ec2456c8ff8b0440c615418"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "bdb489c4e092eef363b61371c2e6d11e"
server_key_iv: "aab4800708223fd38f14f77d5ab97988"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.628091 17315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "26f9ea9b3ec2456c8ff8b0440c615418"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "bdb489c4e092eef363b61371c2e6d11e"
server_key_iv: "aab4800708223fd38f14f77d5ab97988"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.629792 17315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:30.630702 17342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.630937 17315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:30.631055 17315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "26f9ea9b3ec2456c8ff8b0440c615418"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-x5fp"
server_key: "bdb489c4e092eef363b61371c2e6d11e"
server_key_iv: "aab4800708223fd38f14f77d5ab97988"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.631148 17315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:30.647035 17315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:30.649735 17315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:30.649919 17315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:30.650588 17315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:30.650640 17315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.650686 17315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:30.650744 17315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:30.658987 17315 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:46265
I20260810 07:19:30.659076 17520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:46265 every 8 connection(s)
I20260810 07:19:30.659425 17315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:30.661203 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17315
I20260810 07:19:30.661326 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:30.661517 14992 external_mini_cluster.cc:1428] Setting key 979ea3eecab8c4d9499c395be8ccfb34
I20260810 07:19:30.662209 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.651482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:37099 (local address 127.14.164.62:44631)
0810 07:19:30.651567 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.651569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.651582 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:30.652512 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.652513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.652525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.652576 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.653683 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.654553 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.655665 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.655881 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.655974 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.658723 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.658725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.658725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.658734 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.660434 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.660927 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.660928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.660929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.660988 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.661305 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.661306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.661307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.661474 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.661551 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.661989 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.662091 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:19:30.662413 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.651082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.14.164.3:37099)
0810 07:19:30.651284 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.651343 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.652311 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.652898 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.652903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.653304 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.653591 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.653597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.654665 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.654666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.655546 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.655549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.656000 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.656002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.656138 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.656742 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.656755 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.658618 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.660513 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.660520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.660522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.660857 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.661074 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.661075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.661075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.661197 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.661605 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.661614 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.661817 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.661929 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.662129 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":127,"threads_started":1}
I20260810 07:19:30.662876 17521 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44631
I20260810 07:19:30.662984 17521 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:30.663180 17521 heartbeater.cc:511] Master 127.14.164.62:44631 requested a full tablet report, sending...
I20260810 07:19:30.663573 16745 ts_manager.cc:194] Registered new tserver with Master: 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3:46265)
I20260810 07:19:30.664088 16745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:37099
I20260810 07:19:30.670004 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.663935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:55199 (local address 127.14.164.62:44631)
0810 07:19:30.663980 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.663983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.664078 (+ 95us) server_negotiation.cc:407] Connection header received
0810 07:19:30.664152 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.664152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.664162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.664205 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.664635 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.665255 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.666025 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.666232 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.666291 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.667238 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.667239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.667240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.667246 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.668931 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.669284 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.669284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.669285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.669339 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.669542 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.669543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.669543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.669662 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.669715 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.669879 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.669939 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:30.670022 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.663872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.14.164.3:55199)
0810 07:19:30.664000 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.664011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.664094 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.664247 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.664248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.664472 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.664557 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.664559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.665407 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.665408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.665943 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.665944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.666308 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.666309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.666377 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.666841 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.666845 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.667137 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.669026 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.669027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.669027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.669206 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.669398 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.669399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.669399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.669469 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.669721 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.669722 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.669793 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.669857 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.669952 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":8,"thread_start_us":69,"threads_started":1}
I20260810 07:19:30.675145 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:30.702732 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.693858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.14.164.62:44631)
0810 07:19:30.693934 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.693936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.693984 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:19:30.694247 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.694248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.694263 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.694334 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.695000 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.695662 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.696680 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.696949 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.697075 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.699920 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.699922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.699922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.699933 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.701487 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.701891 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.701892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.701892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.701960 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.702180 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.702181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.702181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.702308 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.702418 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.702533 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.702624 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:30.705025 16745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:30.706722 16745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:30.714177 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.709672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:41435 (local address 127.0.0.1:45622)
0810 07:19:30.709852 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.709899 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.709972 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.710765 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.710769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.710772 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.710896 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.710899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.711821 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.711822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.712894 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.712913 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.713880 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.713881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.713983 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.714003 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.714055 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.714106 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":18,"thread_start_us":63,"threads_started":1}
I20260810 07:19:30.714608 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.709789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.14.164.2:41435)
0810 07:19:30.710218 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.710223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.710236 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:30.710594 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.710597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.710618 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.710872 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.710968 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.711721 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.712990 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.713754 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.714344 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.714430 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.714459 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.714501 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":91,"threads_started":1}
I20260810 07:19:30.715161 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.709734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:46265 (local address 127.0.0.1:38632)
0810 07:19:30.710645 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.710653 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.710697 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.710956 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.710957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.710960 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.711013 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.711015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.712932 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.712933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.713695 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.713697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.714822 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.714823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.714975 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.714980 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.715029 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.715073 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":2,"spinlock_wait_cycles":1280,"thread_start_us":719,"threads_started":1}
I20260810 07:19:30.715178 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.710146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.14.164.1:42987)
0810 07:19:30.710387 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.710391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.710586 (+ 195us) server_negotiation.cc:407] Connection header received
0810 07:19:30.710639 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.710642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.710660 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.710740 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.711357 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.712515 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.713980 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.714728 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.714807 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.714864 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.715026 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.715073 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260810 07:19:30.715459 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.709983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.14.164.3:46265)
0810 07:19:30.710242 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.710248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.710755 (+ 507us) server_negotiation.cc:407] Connection header received
0810 07:19:30.710799 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.710802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.710816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.710884 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.711259 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.712443 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.713810 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.714728 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.714807 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.714864 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.715286 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.715346 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
I20260810 07:19:30.715770 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.709734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:42987 (local address 127.0.0.1:37880)
0810 07:19:30.710499 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.710510 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.710568 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.710826 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.710827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.710830 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.711227 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.711232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.712620 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.712621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.713893 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.713895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.714832 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.714833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.714909 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.714914 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.715628 (+ 714us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.715677 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":1,"thread_start_us":719,"threads_started":1}
I20260810 07:19:30.715584 17200 tablet_service.cc:1467] Processing CreateTablet for tablet 4937fe27ca054baa896656ff478830fb (DEFAULT_TABLE table=test-table [id=a989eab7035a4f14bd32f96deb47bf25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:30.716277 17200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4937fe27ca054baa896656ff478830fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.716221 17422 tablet_service.cc:1467] Processing CreateTablet for tablet 4937fe27ca054baa896656ff478830fb (DEFAULT_TABLE table=test-table [id=a989eab7035a4f14bd32f96deb47bf25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:30.716616 17422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4937fe27ca054baa896656ff478830fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.716818 16964 tablet_service.cc:1467] Processing CreateTablet for tablet 4937fe27ca054baa896656ff478830fb (DEFAULT_TABLE table=test-table [id=a989eab7035a4f14bd32f96deb47bf25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:30.717224 16964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4937fe27ca054baa896656ff478830fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.719568 17556 tablet_bootstrap.cc:492] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Bootstrap starting.
I20260810 07:19:30.720613 17556 tablet_bootstrap.cc:654] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.721086 17556 log.cc:824] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:30.721102 17557 tablet_bootstrap.cc:492] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Bootstrap starting.
I20260810 07:19:30.721904 17557 tablet_bootstrap.cc:654] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.722043 17556 tablet_bootstrap.cc:492] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: No bootstrap required, opened a new log
I20260810 07:19:30.722119 17556 ts_tablet_manager.cc:1397] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:30.722100 17560 tablet_bootstrap.cc:492] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Bootstrap starting.
I20260810 07:19:30.722493 17557 log.cc:824] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:30.722864 17560 tablet_bootstrap.cc:654] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.723254 17560 log.cc:824] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:30.723486 17557 tablet_bootstrap.cc:492] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: No bootstrap required, opened a new log
I20260810 07:19:30.723560 17557 ts_tablet_manager.cc:1397] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:19:30.724066 17560 tablet_bootstrap.cc:492] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: No bootstrap required, opened a new log
I20260810 07:19:30.724162 17560 ts_tablet_manager.cc:1397] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:30.724674 17556 raft_consensus.cc:348] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.724918 17556 raft_consensus.cc:374] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.724972 17556 raft_consensus.cc:729] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f9ea9b3ec2456c8ff8b0440c615418, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.725142 17556 consensus_queue.cc:260] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.725452 17556 ts_tablet_manager.cc:1428] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:30.725483 17521 heartbeater.cc:503] Master 127.14.164.62:44631 was elected leader, sending a full tablet report...
I20260810 07:19:30.725770 17557 raft_consensus.cc:348] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.725968 17557 raft_consensus.cc:374] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.726016 17557 raft_consensus.cc:729] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00378ebf8a84dd5bd4d381f9bdfbbc2, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.726128 17557 consensus_queue.cc:260] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.726419 17557 ts_tablet_manager.cc:1428] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:30.726456 17301 heartbeater.cc:503] Master 127.14.164.62:44631 was elected leader, sending a full tablet report...
I20260810 07:19:30.726468 17560 raft_consensus.cc:348] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.726649 17560 raft_consensus.cc:374] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.726714 17560 raft_consensus.cc:729] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d36ea39c2d42468f741ad44179cbd5, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.726861 17560 consensus_queue.cc:260] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.727190 17560 ts_tablet_manager.cc:1428] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:30.727262 17072 heartbeater.cc:503] Master 127.14.164.62:44631 was elected leader, sending a full tablet report...
W20260810 07:19:30.740525 17302 tablet.cc:2369] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:30.777254 17565 raft_consensus.cc:482] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:30.777407 17565 raft_consensus.cc:504] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.777837 17565 leader_election.cc:288] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3:46265), e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.781163 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.778413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:39903 (local address 127.14.164.2:41435)
0810 07:19:30.778489 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.778490 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.778498 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:30.778524 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.778525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.778532 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.778601 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.778790 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.779306 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.780336 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.780763 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.780854 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.780872 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.781043 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.781083 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:30.781159 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.778306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:46765 (local address 127.14.164.3:46265)
0810 07:19:30.778355 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.778357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.778506 (+ 149us) server_negotiation.cc:407] Connection header received
0810 07:19:30.778596 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.778598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.778613 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.778679 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.778858 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.779324 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.780329 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.780697 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.780836 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.780847 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.781027 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.781068 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:30.781174 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.778190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:46265 (local address 127.14.164.1:46765)
0810 07:19:30.778389 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.778400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.778522 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.778685 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.778686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.778688 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.778782 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.778783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.779421 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.779422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.780229 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.780231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.780852 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.780853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.780945 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.780949 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.781020 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.781065 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":33,"thread_start_us":121,"threads_started":1}
I20260810 07:19:30.781279 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.778180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:41435 (local address 127.14.164.1:39903)
0810 07:19:30.778259 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.778274 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.778343 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.778606 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.778608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.778610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.778709 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.778711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.779415 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.779416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.780230 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.780233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.780851 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.780852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.780956 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.780959 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.781024 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.781068 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":52}
I20260810 07:19:30.781525 17455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4937fe27ca054baa896656ff478830fb" candidate_uuid: "16d36ea39c2d42468f741ad44179cbd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" is_pre_election: true
I20260810 07:19:30.781643 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4937fe27ca054baa896656ff478830fb" candidate_uuid: "16d36ea39c2d42468f741ad44179cbd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" is_pre_election: true
I20260810 07:19:30.781713 17455 raft_consensus.cc:2454] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16d36ea39c2d42468f741ad44179cbd5 in term 0.
I20260810 07:19:30.781795 17232 raft_consensus.cc:2454] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16d36ea39c2d42468f741ad44179cbd5 in term 0.
I20260810 07:19:30.781945 16886 leader_election.cc:302] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d36ea39c2d42468f741ad44179cbd5, 26f9ea9b3ec2456c8ff8b0440c615418; no voters:
I20260810 07:19:30.782099 17565 raft_consensus.cc:2781] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:30.782162 17565 raft_consensus.cc:482] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:30.782194 17565 raft_consensus.cc:3037] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.782902 17565 raft_consensus.cc:504] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.783062 17565 leader_election.cc:288] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [CANDIDATE]: Term 1 election: Requested vote from peers 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3:46265), e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.783226 17455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4937fe27ca054baa896656ff478830fb" candidate_uuid: "16d36ea39c2d42468f741ad44179cbd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f9ea9b3ec2456c8ff8b0440c615418"
I20260810 07:19:30.783242 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4937fe27ca054baa896656ff478830fb" candidate_uuid: "16d36ea39c2d42468f741ad44179cbd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2"
I20260810 07:19:30.783318 17232 raft_consensus.cc:3037] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.783314 17455 raft_consensus.cc:3037] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.784296 17455 raft_consensus.cc:2454] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16d36ea39c2d42468f741ad44179cbd5 in term 1.
I20260810 07:19:30.784296 17232 raft_consensus.cc:2454] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16d36ea39c2d42468f741ad44179cbd5 in term 1.
I20260810 07:19:30.784524 16886 leader_election.cc:302] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d36ea39c2d42468f741ad44179cbd5, 26f9ea9b3ec2456c8ff8b0440c615418; no voters:
I20260810 07:19:30.784632 17565 raft_consensus.cc:2781] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:30.784804 17565 raft_consensus.cc:686] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 1 LEADER]: Becoming Leader. State: Replica: 16d36ea39c2d42468f741ad44179cbd5, State: Running, Role: LEADER
I20260810 07:19:30.784920 17565 consensus_queue.cc:237] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } }
I20260810 07:19:30.785845 16746 catalog_manager.cc:5495] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "16d36ea39c2d42468f741ad44179cbd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } health_report { overall_health: HEALTHY } } }
W20260810 07:19:30.806656 17074 tablet.cc:2369] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:30.811070 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.807642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.14.164.1:42987)
0810 07:19:30.807687 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.807689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.807721 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:30.807814 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.807815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.807833 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.807892 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:30.808128 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.808609 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.809612 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.809839 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.809899 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.810159 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:30.810262 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:30.810339 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.810489 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.810510 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:30.813230 17232 raft_consensus.cc:1261] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Refusing update from remote peer 16d36ea39c2d42468f741ad44179cbd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:30.813228 17455 raft_consensus.cc:1261] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Refusing update from remote peer 16d36ea39c2d42468f741ad44179cbd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:30.813596 17565 consensus_queue.cc:1035] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:30.813851 17568 consensus_queue.cc:1035] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:30.817140 17575 mvcc.cc:205] Tried to move back new op lower bound from 7316874734847602688 to 7316874734735671296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874734847602688}]
I20260810 07:19:30.817312 17577 mvcc.cc:205] Tried to move back new op lower bound from 7316874734847602688 to 7316874734735671296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874734847602688}]
I20260810 07:19:30.818866 17574 mvcc.cc:205] Tried to move back new op lower bound from 7316874734847602688 to 7316874734735671296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874734847602688 or (T in {7316874734847602688})}]
I20260810 07:19:30.820636 16745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:30.820861 16745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:30.821874 16745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: Sending DeleteTablet for 3 replicas of tablet 4937fe27ca054baa896656ff478830fb
I20260810 07:19:30.822338 17200 tablet_service.cc:1514] Processing DeleteTablet for tablet 4937fe27ca054baa896656ff478830fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:30 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:45622
I20260810 07:19:30.822348 16964 tablet_service.cc:1514] Processing DeleteTablet for tablet 4937fe27ca054baa896656ff478830fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:30 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:37880
I20260810 07:19:30.822432 17422 tablet_service.cc:1514] Processing DeleteTablet for tablet 4937fe27ca054baa896656ff478830fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:30 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38632
I20260810 07:19:30.822557 17591 tablet_replica.cc:330] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: stopping tablet replica
I20260810 07:19:30.822662 17591 raft_consensus.cc:2229] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:30.822764 17591 raft_consensus.cc:2258] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:30.822695 16745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:30.822870 16745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:30.823020 17593 tablet_replica.cc:330] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: stopping tablet replica
I20260810 07:19:30.823372 17593 raft_consensus.cc:2229] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:30.823508 17591 ts_tablet_manager.cc:1905] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:30.823639 17593 raft_consensus.cc:2258] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:30.824211 17593 ts_tablet_manager.cc:1905] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:30.824267 16964 tablet_service.cc:1467] Processing CreateTablet for tablet b4f057e477a5444cb0fab26300fb9d61 (DEFAULT_TABLE table=test-table [id=a66b48ac6097444793a5916094724cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:30.824395 16964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f057e477a5444cb0fab26300fb9d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.825285 17200 tablet_service.cc:1467] Processing CreateTablet for tablet b4f057e477a5444cb0fab26300fb9d61 (DEFAULT_TABLE table=test-table [id=a66b48ac6097444793a5916094724cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:30.825433 17200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f057e477a5444cb0fab26300fb9d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.825423 17591 ts_tablet_manager.cc:1918] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:30.825495 17591 log.cc:1196] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/4937fe27ca054baa896656ff478830fb
I20260810 07:19:30.825780 17591 ts_tablet_manager.cc:1939] T 4937fe27ca054baa896656ff478830fb P 16d36ea39c2d42468f741ad44179cbd5: Deleting consensus metadata
I20260810 07:19:30.826289 16726 catalog_manager.cc:4841] TS 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987): tablet 4937fe27ca054baa896656ff478830fb (table test-table [id=a989eab7035a4f14bd32f96deb47bf25]) successfully deleted
I20260810 07:19:30.826921 17593 ts_tablet_manager.cc:1918] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:30.826948 17557 tablet_bootstrap.cc:492] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Bootstrap starting.
I20260810 07:19:30.827167 17593 log.cc:1196] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/4937fe27ca054baa896656ff478830fb
I20260810 07:19:30.827349 17422 tablet_service.cc:1467] Processing CreateTablet for tablet b4f057e477a5444cb0fab26300fb9d61 (DEFAULT_TABLE table=test-table [id=a66b48ac6097444793a5916094724cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:30.827412 17557 tablet_bootstrap.cc:654] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.827500 17422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f057e477a5444cb0fab26300fb9d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.828388 17560 tablet_bootstrap.cc:492] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Bootstrap starting.
I20260810 07:19:30.828773 17556 tablet_bootstrap.cc:492] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Bootstrap starting.
I20260810 07:19:30.828848 17560 tablet_bootstrap.cc:654] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.828858 17592 tablet_replica.cc:330] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: stopping tablet replica
I20260810 07:19:30.828984 17592 raft_consensus.cc:2229] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:30.829134 17592 raft_consensus.cc:2258] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:30.829221 17556 tablet_bootstrap.cc:654] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.829459 17593 ts_tablet_manager.cc:1939] T 4937fe27ca054baa896656ff478830fb P 26f9ea9b3ec2456c8ff8b0440c615418: Deleting consensus metadata
I20260810 07:19:30.829876 17592 ts_tablet_manager.cc:1905] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:30.829993 16724 catalog_manager.cc:4841] TS 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3:46265): tablet 4937fe27ca054baa896656ff478830fb (table test-table [id=a989eab7035a4f14bd32f96deb47bf25]) successfully deleted
I20260810 07:19:30.830368 17556 tablet_bootstrap.cc:492] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: No bootstrap required, opened a new log
I20260810 07:19:30.830439 17556 ts_tablet_manager.cc:1397] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:19:30.830606 17556 raft_consensus.cc:348] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.830706 17556 raft_consensus.cc:374] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.830746 17556 raft_consensus.cc:729] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f9ea9b3ec2456c8ff8b0440c615418, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.830799 17556 consensus_queue.cc:260] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.830902 17556 ts_tablet_manager.cc:1428] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.831637 17592 ts_tablet_manager.cc:1918] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:30.831720 17592 log.cc:1196] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/4937fe27ca054baa896656ff478830fb
I20260810 07:19:30.832007 17592 ts_tablet_manager.cc:1939] T 4937fe27ca054baa896656ff478830fb P e00378ebf8a84dd5bd4d381f9bdfbbc2: Deleting consensus metadata
I20260810 07:19:30.832746 16728 catalog_manager.cc:4841] TS e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435): tablet 4937fe27ca054baa896656ff478830fb (table test-table [id=a989eab7035a4f14bd32f96deb47bf25]) successfully deleted
I20260810 07:19:30.837329 17557 tablet_bootstrap.cc:492] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: No bootstrap required, opened a new log
I20260810 07:19:30.837399 17557 ts_tablet_manager.cc:1397] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260810 07:19:30.837571 17557 raft_consensus.cc:348] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.837659 17557 raft_consensus.cc:374] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.837680 17557 raft_consensus.cc:729] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00378ebf8a84dd5bd4d381f9bdfbbc2, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.837725 17557 consensus_queue.cc:260] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.837826 17557 ts_tablet_manager.cc:1428] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:19:30.838310 17560 tablet_bootstrap.cc:492] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: No bootstrap required, opened a new log
I20260810 07:19:30.838406 17560 ts_tablet_manager.cc:1397] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260810 07:19:30.838593 17560 raft_consensus.cc:348] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.838672 17560 raft_consensus.cc:374] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.838692 17560 raft_consensus.cc:729] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d36ea39c2d42468f741ad44179cbd5, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.838737 17560 consensus_queue.cc:260] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.838820 17560 ts_tablet_manager.cc:1428] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.848457 17563 raft_consensus.cc:482] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:30.848584 17563 raft_consensus.cc:504] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.849011 17563 leader_election.cc:288] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987), e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.852413 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.849397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:51605 (local address 127.14.164.1:42987)
0810 07:19:30.849444 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.849446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.849456 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:30.849487 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.849487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.849497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.849563 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.849786 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.850547 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.851610 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.852011 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.852116 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.852127 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.852286 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.852335 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:30.852420 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.849246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:42987 (local address 127.14.164.3:51605)
0810 07:19:30.849308 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.849319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.849362 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.849618 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.849619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.849627 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.849706 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.849708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.850620 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.850621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.851512 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.851515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.852116 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.852117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.852211 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.852216 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.852272 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.852323 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:19:30.852954 16995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f057e477a5444cb0fab26300fb9d61" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d36ea39c2d42468f741ad44179cbd5" is_pre_election: true
I20260810 07:19:30.853137 16995 raft_consensus.cc:2454] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 0.
I20260810 07:19:30.853339 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.849422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:43057 (local address 127.14.164.2:41435)
0810 07:19:30.849538 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.849540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.849742 (+ 202us) server_negotiation.cc:407] Connection header received
0810 07:19:30.849846 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.849848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.849860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.849969 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:30.850145 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.850839 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.852227 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.852868 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.852973 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.852983 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.853198 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.853252 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 07:19:30.853379 17346 leader_election.cc:302] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d36ea39c2d42468f741ad44179cbd5, 26f9ea9b3ec2456c8ff8b0440c615418; no voters:
I20260810 07:19:30.853530 17563 raft_consensus.cc:2781] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:30.853586 17563 raft_consensus.cc:482] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:30.853621 17563 raft_consensus.cc:3037] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.853705 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.849246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:41435 (local address 127.14.164.3:43057)
0810 07:19:30.849661 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.849670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.849756 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.849974 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.849975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.849977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:30.850073 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.850074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.850934 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.850935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.852145 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.852148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.852976 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.852977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.853056 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.853065 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.853502 (+ 437us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.853549 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":169,"thread_start_us":70,"threads_started":1}
I20260810 07:19:30.854583 17563 raft_consensus.cc:504] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.854744 17563 leader_election.cc:288] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 election: Requested vote from peers 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987), e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.854988 16995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f057e477a5444cb0fab26300fb9d61" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d36ea39c2d42468f741ad44179cbd5"
I20260810 07:19:30.854972 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f057e477a5444cb0fab26300fb9d61" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" is_pre_election: true
I20260810 07:19:30.855020 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f057e477a5444cb0fab26300fb9d61" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2"
I20260810 07:19:30.855080 16995 raft_consensus.cc:3037] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.855084 17232 raft_consensus.cc:2454] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 0.
I20260810 07:19:30.855882 16995 raft_consensus.cc:2454] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 1.
I20260810 07:19:30.856088 17346 leader_election.cc:302] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16d36ea39c2d42468f741ad44179cbd5, 26f9ea9b3ec2456c8ff8b0440c615418; no voters: e00378ebf8a84dd5bd4d381f9bdfbbc2
I20260810 07:19:30.856264 17563 raft_consensus.cc:2781] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:30.856484 17563 raft_consensus.cc:686] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 LEADER]: Becoming Leader. State: Replica: 26f9ea9b3ec2456c8ff8b0440c615418, State: Running, Role: LEADER
I20260810 07:19:30.856599 17563 consensus_queue.cc:237] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.857404 16746 catalog_manager.cc:5495] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } health_report { overall_health: HEALTHY } } }
Aug 10 07:19:30 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:30.890236 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.881830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.14.164.62:44631)
0810 07:19:30.881887 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.881890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.882015 (+ 125us) server_negotiation.cc:407] Connection header received
0810 07:19:30.882127 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.882128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.882140 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.882268 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.882819 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.883410 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.884353 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.884561 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.884672 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.887206 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.887207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.887208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:30.887222 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.888791 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.889283 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.889284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.889285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.889353 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:30.889576 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:30.889578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:30.889578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:30.889711 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.889790 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.889983 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.890090 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:19:30.890327 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.881686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44631 (local address 127.0.0.1:33650)
0810 07:19:30.881901 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:30.881913 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:30.882031 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:30.882276 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:30.882278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:30.882654 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:30.882716 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.882719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.883514 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.883515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.884239 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:30.884243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:30.884682 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:30.884684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:30.884815 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:30.885627 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:30.885632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:30.887082 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:30.888932 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.888943 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.888945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.889177 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.889430 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:30.889431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:30.889431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:30.889502 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:30.889794 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:30.889804 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:30.889865 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 07:19:30.889943 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:19:30.890170 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":97,"thread_start_us":92,"threads_started":1}
I20260810 07:19:30.893503 16746 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:33650:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:19:30.893769 16746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:30.895284 16964 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea6b2c7a4ec4da198633f463facad6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef6db90118f34e8d97ab42581a917d45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:30.895357 17200 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea6b2c7a4ec4da198633f463facad6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef6db90118f34e8d97ab42581a917d45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:30.895406 17422 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea6b2c7a4ec4da198633f463facad6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef6db90118f34e8d97ab42581a917d45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:30.895467 16964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea6b2c7a4ec4da198633f463facad6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.895534 17422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea6b2c7a4ec4da198633f463facad6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.895535 17200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea6b2c7a4ec4da198633f463facad6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.897102 17556 tablet_bootstrap.cc:492] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: Bootstrap starting.
I20260810 07:19:30.897246 17560 tablet_bootstrap.cc:492] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: Bootstrap starting.
I20260810 07:19:30.897581 17556 tablet_bootstrap.cc:654] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.897703 17560 tablet_bootstrap.cc:654] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.898033 17557 tablet_bootstrap.cc:492] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: Bootstrap starting.
I20260810 07:19:30.898412 17556 tablet_bootstrap.cc:492] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: No bootstrap required, opened a new log
I20260810 07:19:30.898461 17556 ts_tablet_manager.cc:1397] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:30.898490 17557 tablet_bootstrap.cc:654] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.898635 17556 raft_consensus.cc:348] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.898721 17556 raft_consensus.cc:374] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.898744 17556 raft_consensus.cc:729] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f9ea9b3ec2456c8ff8b0440c615418, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.898803 17556 consensus_queue.cc:260] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.898929 17556 ts_tablet_manager.cc:1428] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.898969 17563 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:30.899080 17563 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:30.899313 17560 tablet_bootstrap.cc:492] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: No bootstrap required, opened a new log
I20260810 07:19:30.899369 17560 ts_tablet_manager.cc:1397] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:30.899526 17560 raft_consensus.cc:348] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.899611 17560 raft_consensus.cc:374] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.899639 17560 raft_consensus.cc:729] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d36ea39c2d42468f741ad44179cbd5, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.899688 17560 consensus_queue.cc:260] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.899817 17568 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:30.899955 17568 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:30.899816 17560 ts_tablet_manager.cc:1428] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.900198 17557 tablet_bootstrap.cc:492] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: No bootstrap required, opened a new log
I20260810 07:19:30.900247 17557 ts_tablet_manager.cc:1397] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:30.900421 17557 raft_consensus.cc:348] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.900506 17557 raft_consensus.cc:374] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.900532 17557 raft_consensus.cc:729] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00378ebf8a84dd5bd4d381f9bdfbbc2, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.900579 17557 consensus_queue.cc:260] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.900671 17557 ts_tablet_manager.cc:1428] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:19:30.900856 17564 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:30.900971 17564 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 07:19:30.910080 17523 tablet.cc:2369] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:30.996034 17563 raft_consensus.cc:482] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:30.996147 17563 raft_consensus.cc:504] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.996366 17563 leader_election.cc:288] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987), e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.996601 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea6b2c7a4ec4da198633f463facad6e" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" is_pre_election: true
I20260810 07:19:30.996608 16995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea6b2c7a4ec4da198633f463facad6e" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d36ea39c2d42468f741ad44179cbd5" is_pre_election: true
I20260810 07:19:30.996723 17232 raft_consensus.cc:2454] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 0.
I20260810 07:19:30.996774 16995 raft_consensus.cc:2454] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 0.
I20260810 07:19:30.997344 17347 leader_election.cc:302] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f9ea9b3ec2456c8ff8b0440c615418, e00378ebf8a84dd5bd4d381f9bdfbbc2; no voters:
I20260810 07:19:30.997470 17563 raft_consensus.cc:2781] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:30.997516 17563 raft_consensus.cc:482] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:30.997548 17563 raft_consensus.cc:3037] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.998332 17563 raft_consensus.cc:504] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:30.998473 17563 leader_election.cc:288] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 election: Requested vote from peers 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987), e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435)
I20260810 07:19:30.998678 16995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea6b2c7a4ec4da198633f463facad6e" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d36ea39c2d42468f741ad44179cbd5"
I20260810 07:19:30.998668 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea6b2c7a4ec4da198633f463facad6e" candidate_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2"
I20260810 07:19:30.998791 17232 raft_consensus.cc:3037] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.998792 16995 raft_consensus.cc:3037] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.999704 17232 raft_consensus.cc:2454] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 1.
I20260810 07:19:30.999701 16995 raft_consensus.cc:2454] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f9ea9b3ec2456c8ff8b0440c615418 in term 1.
I20260810 07:19:30.999900 17346 leader_election.cc:302] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d36ea39c2d42468f741ad44179cbd5, 26f9ea9b3ec2456c8ff8b0440c615418; no voters:
I20260810 07:19:31.000043 17563 raft_consensus.cc:2781] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:31.000123 17563 raft_consensus.cc:686] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 LEADER]: Becoming Leader. State: Replica: 26f9ea9b3ec2456c8ff8b0440c615418, State: Running, Role: LEADER
I20260810 07:19:31.000167 17563 consensus_queue.cc:237] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } }
I20260810 07:19:31.000630 17601 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: TxnStatusTablet state changed. Reason: New leader 26f9ea9b3ec2456c8ff8b0440c615418. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.000757 17601 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:31.001003 17623 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:19:31.001044 16746 catalog_manager.cc:5495] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } health_report { overall_health: HEALTHY } } }
I20260810 07:19:31.001149 17623 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:19:31.021234 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.018064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.14.164.3:46265)
0810 07:19:31.018119 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:31.018122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:31.018174 (+ 52us) server_negotiation.cc:407] Connection header received
0810 07:19:31.018324 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:31.018325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:31.018347 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:31.018423 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:31.018570 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.019364 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.020183 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.020503 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.020574 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.020853 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:31.020983 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:31.021060 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:31.021120 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 07:19:31.021142 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:19:31.021236 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.017988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:46265 (local address 127.0.0.1:38646)
0810 07:19:31.018115 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:31.018134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:31.018228 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:31.018437 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:31.018438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:31.018439 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:31.018501 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:31.018504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.019456 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.019457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:31.020094 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:31.020096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.020598 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.020599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:31.020659 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.020676 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:31.021057 (+ 381us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:31.021067 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 07:19:31.021107 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 07:19:31.021149 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 07:19:31.022668 17232 raft_consensus.cc:1261] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Refusing update from remote peer 26f9ea9b3ec2456c8ff8b0440c615418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:31.022699 16995 raft_consensus.cc:1261] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Refusing update from remote peer 26f9ea9b3ec2456c8ff8b0440c615418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:31.022975 17601 consensus_queue.cc:1035] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:31.023145 17563 consensus_queue.cc:1035] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:31.023885 17564 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: TxnStatusTablet state changed. Reason: New leader 26f9ea9b3ec2456c8ff8b0440c615418. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.023895 17568 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: TxnStatusTablet state changed. Reason: New leader 26f9ea9b3ec2456c8ff8b0440c615418. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.023988 17564 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:31.023990 17568 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:31.024953 17563 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.025044 17563 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:31.025156 17564 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.025244 17564 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P e00378ebf8a84dd5bd4d381f9bdfbbc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:31.025695 17563 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.025797 17563 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 26f9ea9b3ec2456c8ff8b0440c615418: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:31.026575 17568 tablet_replica.cc:439] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 } } peers { permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 } } peers { permanent_uuid: "26f9ea9b3ec2456c8ff8b0440c615418" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 46265 } } }
I20260810 07:19:31.026674 17568 tablet_replica.cc:442] T 2ea6b2c7a4ec4da198633f463facad6e P 16d36ea39c2d42468f741ad44179cbd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:31.029932 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.027661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.14.164.3:46265)
0810 07:19:31.027705 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:31.027707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:31.027717 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:31.027775 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:31.027776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:31.027785 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:31.027849 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:31.027952 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.028422 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.029222 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.029432 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.029519 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.029666 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:31.029701 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:31.029773 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:31.029828 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:19:31.029852 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:31.034838 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.032570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:57665 (local address 127.14.164.3:46265)
0810 07:19:31.032607 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:31.032609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:31.032619 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:31.032682 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:31.032683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:31.032691 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:31.032779 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:31.032877 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.033312 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.034078 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.034355 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.034428 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.034603 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:31.034634 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:31.034683 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:31.034738 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:19:31.034760 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
I20260810 07:19:31.034845 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.032496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:46265 (local address 127.14.164.3:57665)
0810 07:19:31.032579 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:31.032592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:31.032636 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:31.032783 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:31.032784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:31.032785 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:31.032823 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:31.032824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.033408 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.033409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:31.033992 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:31.033993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.034426 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.034427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:31.034492 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.034517 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:31.034682 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:31.034683 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:19:31.034734 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:19:31.034779 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":3}
I20260810 07:19:31.037206 16995 raft_consensus.cc:1261] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Refusing update from remote peer 26f9ea9b3ec2456c8ff8b0440c615418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:31.037221 17232 raft_consensus.cc:3037] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:31.038131 17232 raft_consensus.cc:1261] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Refusing update from remote peer 26f9ea9b3ec2456c8ff8b0440c615418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:31.038239 17601 consensus_queue.cc:1035] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16d36ea39c2d42468f741ad44179cbd5" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 42987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:31.038393 17601 consensus_queue.cc:1035] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00378ebf8a84dd5bd4d381f9bdfbbc2" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:31.049698 16745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:31.049803 16745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:31.050459 16745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a594d9319e80454da14e430fe2b30d40: Sending DeleteTablet for 3 replicas of tablet b4f057e477a5444cb0fab26300fb9d61
I20260810 07:19:31.050716 16964 tablet_service.cc:1514] Processing DeleteTablet for tablet b4f057e477a5444cb0fab26300fb9d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:31 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:37880
I20260810 07:19:31.050747 17200 tablet_service.cc:1514] Processing DeleteTablet for tablet b4f057e477a5444cb0fab26300fb9d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:31 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:45622
I20260810 07:19:31.050839 17591 tablet_replica.cc:330] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: stopping tablet replica
I20260810 07:19:31.050892 17591 raft_consensus.cc:2229] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:31.050901 17422 tablet_service.cc:1514] Processing DeleteTablet for tablet b4f057e477a5444cb0fab26300fb9d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:31 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38632
I20260810 07:19:31.050923 17591 raft_consensus.cc:2258] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:31.050989 17593 tablet_replica.cc:330] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: stopping tablet replica
I20260810 07:19:31.050999 17592 tablet_replica.cc:330] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: stopping tablet replica
I20260810 07:19:31.051054 17593 raft_consensus.cc:2229] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:31.051054 17592 raft_consensus.cc:2229] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:31.051095 17592 raft_consensus.cc:2258] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:31.051118 17593 raft_consensus.cc:2258] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:31.051401 17591 ts_tablet_manager.cc:1905] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:31.051471 17593 ts_tablet_manager.cc:1905] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:31.052182 17592 ts_tablet_manager.cc:1905] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:31.053175 17591 ts_tablet_manager.cc:1918] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:31.053241 17591 log.cc:1196] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/b4f057e477a5444cb0fab26300fb9d61
I20260810 07:19:31.053269 17593 ts_tablet_manager.cc:1918] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:31.053339 17593 log.cc:1196] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/b4f057e477a5444cb0fab26300fb9d61
I20260810 07:19:31.053470 17591 ts_tablet_manager.cc:1939] T b4f057e477a5444cb0fab26300fb9d61 P 16d36ea39c2d42468f741ad44179cbd5: Deleting consensus metadata
I20260810 07:19:31.053568 17593 ts_tablet_manager.cc:1939] T b4f057e477a5444cb0fab26300fb9d61 P 26f9ea9b3ec2456c8ff8b0440c615418: Deleting consensus metadata
I20260810 07:19:31.053905 17592 ts_tablet_manager.cc:1918] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:31.053963 17592 log.cc:1196] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/b4f057e477a5444cb0fab26300fb9d61
I20260810 07:19:31.054143 17592 ts_tablet_manager.cc:1939] T b4f057e477a5444cb0fab26300fb9d61 P e00378ebf8a84dd5bd4d381f9bdfbbc2: Deleting consensus metadata
I20260810 07:19:31.054553 16726 catalog_manager.cc:4841] TS 16d36ea39c2d42468f741ad44179cbd5 (127.14.164.1:42987): tablet b4f057e477a5444cb0fab26300fb9d61 (table test-table [id=a66b48ac6097444793a5916094724cef]) successfully deleted
I20260810 07:19:31.054721 16728 catalog_manager.cc:4841] TS e00378ebf8a84dd5bd4d381f9bdfbbc2 (127.14.164.2:41435): tablet b4f057e477a5444cb0fab26300fb9d61 (table test-table [id=a66b48ac6097444793a5916094724cef]) successfully deleted
I20260810 07:19:31.055001 16724 catalog_manager.cc:4841] TS 26f9ea9b3ec2456c8ff8b0440c615418 (127.14.164.3:46265): tablet b4f057e477a5444cb0fab26300fb9d61 (table test-table [id=a66b48ac6097444793a5916094724cef]) successfully deleted
Aug 10 07:19:31 dist-test-slave-x5fp krb5kdc[16671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:31.066641 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.054637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.14.164.1:42987)
0810 07:19:31.054682 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:31.054684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:31.056936 (+ 2252us) server_negotiation.cc:407] Connection header received
0810 07:19:31.057017 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:31.057018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:31.057031 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:31.057094 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:31.057830 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.058398 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.059378 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.059575 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.059641 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.063225 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:31.063237 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:31.063237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:31.063247 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:31.065277 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:31.065672 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:31.065676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:31.065677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:31.065748 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:31.065970 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:31.065971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:31.065972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:31.066168 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:31.066321 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:31.066449 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 07:19:31.066541 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:19:31.067034 17062 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37896
I20260810 07:19:31.074491 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:31.068166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.14.164.62:44631)
0810 07:19:31.068211 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:31.068213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:31.068222 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:31.068328 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:31.068330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:31.068342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:31.068416 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:31.069101 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.069606 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.070313 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:31.070514 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:31.070593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:31.071792 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:31.071794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:31.071795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:31.071802 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:31.073289 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:31.073643 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:31.073644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:31.073645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:31.073748 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:31.073958 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:31.073959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:31.073959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:31.074075 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:31.074201 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:31.074299 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 07:19:31.074382 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 07:19:31.074738 16745 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33660
I20260810 07:19:31.075467 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16861
I20260810 07:19:31.094224 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17092
I20260810 07:19:31.111726 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17315
I20260810 07:19:31.127182 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16695
2026-08-10T07:19:31Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2559 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:31 dist-test-slave-x5fp krb5kdc[17649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:31 dist-test-slave-x5fp krb5kdc[17649](info): set up 2 sockets
Aug 10 07:19:31 dist-test-slave-x5fp krb5kdc[17649](info): commencing operation
krb5kdc: starting...
W20260810 07:19:32.405524 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:32 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:32.540676 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:45167
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:45167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:32.627621 17677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:32.627897 17677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:32.627939 17677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:32.629173 17677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:32.629281 17677 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:19:32.629318 17677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:32.629346 17677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:32.629374 17677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:32.629400 17677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:32.630990 17677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:45167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:45167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:32.631297 17677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:32.631544 17677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:32.637653 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:32.637653 17686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:32.638011 17685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:32.638075 17677 server_base.cc:1034] running on GCE node
I20260810 07:19:32.638424 17677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:32.638800 17677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:32.640017 17677 hybrid_clock.cc:648] HybridClock initialized: now 1786346372639965 us; error 67 us; skew 500 ppm
Aug 10 07:19:32 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:32.642982 17677 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:32.643548 17677 webserver.cc:460] Webserver started at http://127.14.164.62:42085/ using document root <none> and password file <none>
I20260810 07:19:32.643800 17677 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:32.643874 17677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:32.644035 17677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:32.645273 17677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "f3a38ea3400d473487a092bff0c0072a"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-x5fp"
server_key: "3550c36d7031f4682454a0cf41d9790e"
server_key_iv: "cad1240f90f551b0fc5e5ab170765b00"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.645681 17677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "f3a38ea3400d473487a092bff0c0072a"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-x5fp"
server_key: "3550c36d7031f4682454a0cf41d9790e"
server_key_iv: "cad1240f90f551b0fc5e5ab170765b00"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.647197 17677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:32.648056 17702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:32.648289 17677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:32.648422 17677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "f3a38ea3400d473487a092bff0c0072a"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-x5fp"
server_key: "3550c36d7031f4682454a0cf41d9790e"
server_key_iv: "cad1240f90f551b0fc5e5ab170765b00"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.648506 17677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:32.670687 17677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:32.673125 17677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:32.673286 17677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:32.678226 17677 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:45167
I20260810 07:19:32.678275 17798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:45167 every 8 connection(s)
I20260810 07:19:32.678617 17677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:32.679286 17799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:32.681965 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Bootstrap starting.
I20260810 07:19:32.682655 17799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:32.682996 17799 log.cc:824] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:32.683791 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: No bootstrap required, opened a new log
I20260810 07:19:32.686203 17799 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a38ea3400d473487a092bff0c0072a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45167 } }
I20260810 07:19:32.686419 17799 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:32.686473 17799 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a38ea3400d473487a092bff0c0072a, State: Initialized, Role: FOLLOWER
I20260810 07:19:32.686645 17799 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a38ea3400d473487a092bff0c0072a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45167 } }
I20260810 07:19:32.686746 17799 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:32.686796 17799 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:32.686846 17799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:32.687481 17799 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a38ea3400d473487a092bff0c0072a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45167 } }
I20260810 07:19:32.687604 17799 leader_election.cc:302] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3a38ea3400d473487a092bff0c0072a; no voters:
I20260810 07:19:32.687801 17799 leader_election.cc:288] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:32.687951 17803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:32.688174 17803 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [term 1 LEADER]: Becoming Leader. State: Replica: f3a38ea3400d473487a092bff0c0072a, State: Running, Role: LEADER
I20260810 07:19:32.688206 17799 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:32.688272 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17677
I20260810 07:19:32.688297 17803 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a38ea3400d473487a092bff0c0072a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45167 } }
I20260810 07:19:32.688369 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:32.688586 14992 external_mini_cluster.cc:1428] Setting key 1f7ae9475a1bde420e7e8ae56bf35324
I20260810 07:19:32.688711 17805 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a38ea3400d473487a092bff0c0072a. Latest consensus state: current_term: 1 leader_uuid: "f3a38ea3400d473487a092bff0c0072a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a38ea3400d473487a092bff0c0072a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45167 } } }
I20260810 07:19:32.688870 17804 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3a38ea3400d473487a092bff0c0072a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a38ea3400d473487a092bff0c0072a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45167 } } }
I20260810 07:19:32.688932 17805 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:32.688946 17804 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:32.690408 17834 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:32.690482 17834 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:32.690768 17821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:32.690959 17821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:32.692705 17821 catalog_manager.cc:1347] Generated new cluster ID: 516d603a05ad492ca77f23279dd0a05f
I20260810 07:19:32.692806 17821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:19:32 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:32.699316 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.689278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48034 (local address 127.14.164.62:45167)
0810 07:19:32.689458 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:32.689470 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:32.689666 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:19:32.689796 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:32.689804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:32.689863 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:32.690011 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:32.690787 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.691944 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.692926 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.693352 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.693481 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.695860 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:32.695864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:32.695865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:32.695878 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:32.697636 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.698058 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.698061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.698061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.698167 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.698394 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.698395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.698395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.698642 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:32.698811 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:32.698914 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:19:32.699059 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260810 07:19:32.715451 17821 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:32.716143 17821 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:32.722128 17821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Generated new TSK 0
I20260810 07:19:32.722386 17821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:32.763587 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45167
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:32.849753 17843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:32.849985 17843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:32.850030 17843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:32.851279 17843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:32.851370 17843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:32.851409 17843 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:19:32.851435 17843 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:32.851465 17843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:32.853168 17843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:32.853425 17843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:32.853681 17843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:32.858805 17858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:32.858881 17856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:32.858819 17854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:32.859016 17843 server_base.cc:1034] running on GCE node
I20260810 07:19:32.859297 17843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:32.859555 17843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:32.860788 17843 hybrid_clock.cc:648] HybridClock initialized: now 1786346372860768 us; error 46 us; skew 500 ppm
Aug 10 07:19:32 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:32.863692 17843 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:32.864245 17843 webserver.cc:460] Webserver started at http://127.14.164.1:37163/ using document root <none> and password file <none>
I20260810 07:19:32.864513 17843 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:32.864588 17843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:32.864805 17843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:32.865903 17843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "cc0e0a7df56e4d4eb4195ef177154727"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-x5fp"
server_key: "f5c56b3894909350269869f1d0a61da6"
server_key_iv: "a3d5f1d9273ccba0b92e1efc5c66bb78"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.866283 17843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "cc0e0a7df56e4d4eb4195ef177154727"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-x5fp"
server_key: "f5c56b3894909350269869f1d0a61da6"
server_key_iv: "a3d5f1d9273ccba0b92e1efc5c66bb78"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.867699 17843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:32.868508 17867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:32.868706 17843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:32.868851 17843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "cc0e0a7df56e4d4eb4195ef177154727"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-x5fp"
server_key: "f5c56b3894909350269869f1d0a61da6"
server_key_iv: "a3d5f1d9273ccba0b92e1efc5c66bb78"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.868944 17843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:32.878734 17843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:32.881752 17843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:32.881950 17843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:32.882550 17843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:32.882620 17843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:32.882668 17843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:32.882699 17843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:32 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:32.890618 17843 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:38557
I20260810 07:19:32.890734 18047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:38557 every 8 connection(s)
I20260810 07:19:32.891052 17843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:32.893666 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.883126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:56989 (local address 127.14.164.62:45167)
0810 07:19:32.883171 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:32.883173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:32.883448 (+ 275us) server_negotiation.cc:407] Connection header received
0810 07:19:32.884043 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:32.884045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:32.884065 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:32.884138 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:32.885207 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.885750 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.886820 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.887040 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.887116 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.890516 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:32.890518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:32.890519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:32.890530 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:32.892207 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.892662 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.892663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.892663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.892715 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.893022 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.893023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.893023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.893136 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:32.893191 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:32.893526 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 07:19:32.893597 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:32.893918 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.883118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.14.164.1:56989)
0810 07:19:32.883321 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:32.883358 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:32.883918 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:32.884270 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:32.884276 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:32.884735 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:32.885073 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:32.885081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.885854 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.885855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:32.886706 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:32.886709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.887126 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.887128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:32.887283 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.887835 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:32.887844 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:32.890362 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:32.892317 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:32.892324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:32.892326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:32.892587 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:32.892843 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:32.892844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:32.892844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:32.892963 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:32.893227 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:32.893234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:32.893387 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 07:19:32.893500 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:19:32.893648 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":122,"threads_started":1}
I20260810 07:19:32.894486 18052 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45167
I20260810 07:19:32.894573 18052 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:32.894770 18052 heartbeater.cc:511] Master 127.14.164.62:45167 requested a full tablet report, sending...
I20260810 07:19:32.895409 17728 ts_manager.cc:194] Registered new tserver with Master: cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557)
I20260810 07:19:32.896317 17728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:56989
I20260810 07:19:32.899943 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17843
I20260810 07:19:32.900048 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:32.900251 14992 external_mini_cluster.cc:1428] Setting key dfef4112bebab97a0cb243dbfa8c378c
I20260810 07:19:32.902796 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.895847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:39019 (local address 127.14.164.62:45167)
0810 07:19:32.895895 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:32.895898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:32.895952 (+ 54us) server_negotiation.cc:407] Connection header received
0810 07:19:32.896011 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:32.896011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:32.896020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:32.896073 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:32.896478 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.897163 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.897957 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.898144 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.898216 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.899372 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:32.899374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:32.899374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:32.899381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:32.901448 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.901877 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.901878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.901879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.901948 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.902178 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.902179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.902179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.902308 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:32.902388 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:32.902601 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 07:19:32.902687 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:19:32.902812 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.895762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.14.164.1:39019)
0810 07:19:32.895892 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:32.895902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:32.895955 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:32.896076 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:32.896077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:32.896270 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:32.896346 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:32.896348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.897258 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.897259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:32.897858 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:32.897860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.898264 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.898265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:32.898339 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.898868 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:32.898878 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:32.899225 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:32.901563 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:32.901564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:32.901565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:32.901771 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:32.902026 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:32.902027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:32.902028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:32.902103 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:32.902398 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:32.902399 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:32.902478 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 07:19:32.902554 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:19:32.902685 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:32.932245 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45167
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:33.019039 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:33.019285 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:33.019316 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:33.020653 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:33.020711 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:33.020807 18071 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:19:33.020823 18071 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:33.020844 18071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:33.022485 18071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:33.022722 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:33.022938 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:33.028827 18079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:33.028798 18082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:33.028996 18071 server_base.cc:1034] running on GCE node
W20260810 07:19:33.028795 18078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:33.029268 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:33.029538 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:33.030702 18071 hybrid_clock.cc:648] HybridClock initialized: now 1786346373030690 us; error 43 us; skew 500 ppm
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:33.033332 18071 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:33.033790 18071 webserver.cc:460] Webserver started at http://127.14.164.2:33373/ using document root <none> and password file <none>
I20260810 07:19:33.034075 18071 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:33.034147 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:33.034298 18071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:33.035360 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "0f30a1d6b52c4426b87f27454efd143c"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-x5fp"
server_key: "8af254c704a8e23c1e4ca43654e37a8e"
server_key_iv: "a8562fd845304ba17e4cc7f6b22b267c"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.035763 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "0f30a1d6b52c4426b87f27454efd143c"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-x5fp"
server_key: "8af254c704a8e23c1e4ca43654e37a8e"
server_key_iv: "a8562fd845304ba17e4cc7f6b22b267c"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.037214 18071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:33.038033 18098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.038262 18071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:33.038357 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "0f30a1d6b52c4426b87f27454efd143c"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-x5fp"
server_key: "8af254c704a8e23c1e4ca43654e37a8e"
server_key_iv: "a8562fd845304ba17e4cc7f6b22b267c"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.038416 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:33.048619 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:33.051198 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:33.051337 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:33.051952 18071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:33.052026 18071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.052076 18071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:33.052099 18071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.059155 18071 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:37449
I20260810 07:19:33.059294 18277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:37449 every 8 connection(s)
I20260810 07:19:33.059594 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:33.063825 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.052673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:41367 (local address 127.14.164.62:45167)
0810 07:19:33.052782 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.052784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.053107 (+ 323us) server_negotiation.cc:407] Connection header received
0810 07:19:33.053829 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.053831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.053845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.053943 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.054926 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.055829 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.057547 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.057888 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.057976 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.060909 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.060911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.060911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:33.060918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.062237 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.062685 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.062687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.062687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.062760 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.063099 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.063100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.063100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.063236 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.063296 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.063606 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.063695 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 07:19:33.063861 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.052669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.14.164.2:41367)
0810 07:19:33.052954 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.053001 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.053694 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.054038 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.054043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.054465 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:33.054805 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.054812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.055949 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.055950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.057447 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.057450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.057991 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.057993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.058167 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.058902 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:33.058913 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:33.060821 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:33.062333 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.062340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.062342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.062572 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.062865 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.062866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.062866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.062984 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.063302 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:33.063308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:33.063431 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.063515 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.063647 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":209,"threads_started":1}
I20260810 07:19:33.064492 18278 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45167
I20260810 07:19:33.064590 18278 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:33.064893 18278 heartbeater.cc:511] Master 127.14.164.62:45167 requested a full tablet report, sending...
I20260810 07:19:33.065354 17728 ts_manager.cc:194] Registered new tserver with Master: 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449)
I20260810 07:19:33.065913 17728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:41367
I20260810 07:19:33.068681 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18071
I20260810 07:19:33.068873 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:33.069077 14992 external_mini_cluster.cc:1428] Setting key a0d87eed2e82c81634668e1c7ec950a4
I20260810 07:19:33.072326 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.065879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:35019 (local address 127.14.164.62:45167)
0810 07:19:33.065926 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.065928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.066062 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:19:33.066160 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.066161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.066172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.066268 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.066673 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.067242 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.067961 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.068154 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.068221 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.069387 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.069389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.069390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:33.069397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.071069 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.071399 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.071400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.071401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.071467 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.071747 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.071748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.071748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.071962 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.072020 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.072200 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.072265 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:33.072484 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.065795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.14.164.2:35019)
0810 07:19:33.065980 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.065991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.066091 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.066304 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.066305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.066501 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:33.066582 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.066583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.067352 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.067353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.067867 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.067868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.068233 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.068234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.068293 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.068854 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:33.068860 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:33.069288 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:33.071161 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.071162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.071162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.071329 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.071534 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.071554 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.071555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.071632 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.072037 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:33.072038 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:33.072102 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.072184 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.072302 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:33.100472 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45167
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:33.189009 18296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:33.189281 18296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:33.189309 18296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:33.190668 18296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:33.190716 18296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:33.190737 18296 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:19:33.190775 18296 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:33.190810 18296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:33.192414 18296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:33.192682 18296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:33.192960 18296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:33.198410 18313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:33.198431 18310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:33.198431 18307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:33.198411 18296 server_base.cc:1034] running on GCE node
I20260810 07:19:33.198952 18296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:33.199226 18296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:33.200402 18296 hybrid_clock.cc:648] HybridClock initialized: now 1786346373200374 us; error 45 us; skew 500 ppm
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:33.203166 18296 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:33.203678 18296 webserver.cc:460] Webserver started at http://127.14.164.3:34141/ using document root <none> and password file <none>
I20260810 07:19:33.203934 18296 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:33.204006 18296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:33.204166 18296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:33.205399 18296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "c034d0cdedab4d848c8b66689955d174"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-x5fp"
server_key: "8e99fae561891c3be81e84b38b2d1d50"
server_key_iv: "61a80a48f26c19d421f11beaa4112315"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.205798 18296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "c034d0cdedab4d848c8b66689955d174"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-x5fp"
server_key: "8e99fae561891c3be81e84b38b2d1d50"
server_key_iv: "61a80a48f26c19d421f11beaa4112315"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.207224 18296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:19:33.208232 18324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.208453 18296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:33.208576 18296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "c034d0cdedab4d848c8b66689955d174"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-x5fp"
server_key: "8e99fae561891c3be81e84b38b2d1d50"
server_key_iv: "61a80a48f26c19d421f11beaa4112315"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.208658 18296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:33.224633 18296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:33.226970 18296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:33.227167 18296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:33.227808 18296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:33.227914 18296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.227953 18296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:33.227988 18296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:33.235369 18296 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:42867
I20260810 07:19:33.235477 18508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:42867 every 8 connection(s)
I20260810 07:19:33.235832 18296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:33.237332 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18296
I20260810 07:19:33.237463 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:33.237670 14992 external_mini_cluster.cc:1428] Setting key a4b3d0cf4ba33611c234ae99a107377a
I20260810 07:19:33.238991 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.228303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:36543 (local address 127.14.164.62:45167)
0810 07:19:33.228385 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.228387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.228649 (+ 262us) server_negotiation.cc:407] Connection header received
0810 07:19:33.229250 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.229252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.229267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.229366 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.230068 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.230907 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.232528 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.232892 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.232979 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.235534 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.235536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.235537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:33.235545 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.237298 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.237888 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.237890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.237890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.237960 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.238269 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.238270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.238270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.238416 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.238497 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.238806 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.238897 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:33.239176 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.228312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.14.164.3:36543)
0810 07:19:33.228496 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.228539 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.229136 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.229421 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.229424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.229735 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:33.229950 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.229955 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.231015 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.231016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.232402 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.232406 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.233001 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.233002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.233146 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.233686 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:33.233694 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:33.235420 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:33.237399 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.237408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.237410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.237743 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.238040 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.238041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.238041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.238201 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.238488 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:33.238495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:33.238658 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.238766 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.238946 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260810 07:19:33.239629 18509 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45167
I20260810 07:19:33.239717 18509 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:33.239871 18509 heartbeater.cc:511] Master 127.14.164.62:45167 requested a full tablet report, sending...
I20260810 07:19:33.240257 17728 ts_manager.cc:194] Registered new tserver with Master: c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:33.240624 17728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:36543
I20260810 07:19:33.246541 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.241022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:38291 (local address 127.14.164.62:45167)
0810 07:19:33.241202 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.241205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.241216 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:33.241237 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.241238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.241247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.241311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.241616 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.242105 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.242963 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.243138 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.243190 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.244082 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.244083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.244083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:33.244089 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.245574 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.245901 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.245901 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.245901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.245950 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.246116 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.246117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.246117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.246209 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.246257 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.246423 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.246481 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260810 07:19:33.246546 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.240816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.14.164.3:38291)
0810 07:19:33.240989 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.241002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.241054 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.241316 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.241317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.241473 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:33.241538 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.241540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.242309 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.242310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.242885 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.242887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.243227 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.243228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.243292 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.243692 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:33.243697 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:33.244002 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:33.245654 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.245655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.245655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.245833 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.246008 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.246009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.246009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.246065 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.246274 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:33.246275 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:33.246328 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.246390 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.246472 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260810 07:19:33.249493 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:33.275883 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.267676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.14.164.62:45167)
0810 07:19:33.267750 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.267752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.267810 (+ 58us) server_negotiation.cc:407] Connection header received
0810 07:19:33.267868 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.267869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.267879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.267935 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.268628 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.269256 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.270121 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.270369 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.270441 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.273181 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.273183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.273183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:33.273200 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.274659 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.275055 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.275057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.275058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.275121 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.275338 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.275339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.275340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.275464 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.275580 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.275687 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.275774 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:33.277932 17728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:33.279486 17728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:33.286317 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.282248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:42867 (local address 127.0.0.1:59888)
0810 07:19:33.282399 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.282409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.282456 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.283020 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.283020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.283022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.283109 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.283110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.284129 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.284130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.285167 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.285170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.285983 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.285984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.286065 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.286081 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.286160 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.286227 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":79,"thread_start_us":88,"threads_started":1}
I20260810 07:19:33.286338 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.282489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.14.164.3:42867)
0810 07:19:33.282849 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.282853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.282862 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:33.282909 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.282911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.282926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.283008 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.283213 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.283962 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.285258 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.285862 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.285982 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.286048 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.286182 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.286223 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":104,"threads_started":1}
I20260810 07:19:33.286803 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.281966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:37449 (local address 127.0.0.1:38368)
0810 07:19:33.282154 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.282198 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.282264 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.282622 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.282626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.282628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.282697 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.282698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.284048 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.284050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.285593 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.285597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.286556 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.286557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.286624 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.286629 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.286681 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.286724 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":36,"thread_start_us":78,"threads_started":1}
I20260810 07:19:33.287096 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.282088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.14.164.2:37449)
0810 07:19:33.282464 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.282468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.282479 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:33.282524 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.282526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.282542 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.282619 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.282770 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.283945 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.285706 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.286476 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.286542 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.286594 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.286941 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.286989 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":92,"threads_started":1}
I20260810 07:19:33.287441 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.281988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38557 (local address 127.0.0.1:51568)
0810 07:19:33.282386 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.282398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.282496 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.283018 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.283019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.283022 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.283098 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.283100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.284276 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.284277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.285848 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.285851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.287179 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.287180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.287250 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.287254 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.287317 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.287361 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":14,"thread_start_us":106,"threads_started":1}
I20260810 07:19:33.287348 18399 tablet_service.cc:1467] Processing CreateTablet for tablet aa4f8fbf9fff4693be5ac457bff36d0c (DEFAULT_TABLE table=test-table [id=33a2e9b36aa547549595a1a5e6d17f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:33.287655 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.282110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.14.164.1:38557)
0810 07:19:33.282324 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.282329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.282462 (+ 133us) server_negotiation.cc:407] Connection header received
0810 07:19:33.282812 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.282815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.282834 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.282916 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.283186 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.284195 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.285943 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.287010 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.287084 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.287139 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.287337 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.287382 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260810 07:19:33.287708 18184 tablet_service.cc:1467] Processing CreateTablet for tablet aa4f8fbf9fff4693be5ac457bff36d0c (DEFAULT_TABLE table=test-table [id=33a2e9b36aa547549595a1a5e6d17f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:33.287850 18399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa4f8fbf9fff4693be5ac457bff36d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.288122 18184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa4f8fbf9fff4693be5ac457bff36d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.288264 17957 tablet_service.cc:1467] Processing CreateTablet for tablet aa4f8fbf9fff4693be5ac457bff36d0c (DEFAULT_TABLE table=test-table [id=33a2e9b36aa547549595a1a5e6d17f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:33.288594 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa4f8fbf9fff4693be5ac457bff36d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.291076 18541 tablet_bootstrap.cc:492] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Bootstrap starting.
I20260810 07:19:33.292140 18541 tablet_bootstrap.cc:654] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.292523 18541 log.cc:824] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:33.292632 18544 tablet_bootstrap.cc:492] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Bootstrap starting.
I20260810 07:19:33.292938 18543 tablet_bootstrap.cc:492] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Bootstrap starting.
I20260810 07:19:33.293522 18544 tablet_bootstrap.cc:654] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.293672 18543 tablet_bootstrap.cc:654] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.293913 18544 log.cc:824] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:33.294035 18543 log.cc:824] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:33.294770 18544 tablet_bootstrap.cc:492] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: No bootstrap required, opened a new log
I20260810 07:19:33.294795 18541 tablet_bootstrap.cc:492] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: No bootstrap required, opened a new log
I20260810 07:19:33.294862 18541 ts_tablet_manager.cc:1397] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260810 07:19:33.294862 18544 ts_tablet_manager.cc:1397] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:33.295328 18543 tablet_bootstrap.cc:492] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: No bootstrap required, opened a new log
I20260810 07:19:33.295471 18543 ts_tablet_manager.cc:1397] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:33.296597 18544 raft_consensus.cc:348] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.296813 18544 raft_consensus.cc:374] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.296867 18544 raft_consensus.cc:729] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.296993 18544 consensus_queue.cc:260] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.297065 18541 raft_consensus.cc:348] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.297238 18541 raft_consensus.cc:374] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.297286 18541 raft_consensus.cc:729] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.297348 18544 ts_tablet_manager.cc:1428] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:33.297412 18541 consensus_queue.cc:260] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.297488 18052 heartbeater.cc:503] Master 127.14.164.62:45167 was elected leader, sending a full tablet report...
I20260810 07:19:33.297521 18543 raft_consensus.cc:348] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.297688 18541 ts_tablet_manager.cc:1428] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260810 07:19:33.297681 18543 raft_consensus.cc:374] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.297726 18543 raft_consensus.cc:729] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f30a1d6b52c4426b87f27454efd143c, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.297828 18509 heartbeater.cc:503] Master 127.14.164.62:45167 was elected leader, sending a full tablet report...
I20260810 07:19:33.297873 18543 consensus_queue.cc:260] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.298183 18543 ts_tablet_manager.cc:1428] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:19:33.298318 18278 heartbeater.cc:503] Master 127.14.164.62:45167 was elected leader, sending a full tablet report...
W20260810 07:19:33.310297 18279 tablet.cc:2369] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:33.343355 18551 raft_consensus.cc:482] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:33.343497 18551 raft_consensus.cc:504] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.343950 18551 leader_election.cc:288] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449), c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:33.348546 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.344290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:47873 (local address 127.14.164.3:42867)
0810 07:19:33.344336 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.344338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.344350 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:33.344492 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.344493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.344505 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.344572 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.344849 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.345618 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.347443 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.348104 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.348192 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.348203 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.348413 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.348462 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:19:33.348548 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.344266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:57193 (local address 127.14.164.2:37449)
0810 07:19:33.344314 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.344317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.344486 (+ 169us) server_negotiation.cc:407] Connection header received
0810 07:19:33.344523 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.344524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.344533 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.344587 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.344849 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.345599 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.347443 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.348095 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.348192 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.348203 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.348413 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.348463 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:19:33.348546 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.344186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:42867 (local address 127.14.164.1:47873)
0810 07:19:33.344265 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.344280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.344345 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.344616 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.344617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.344619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.344757 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.344759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.345702 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.345703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.347307 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.347312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.348217 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.348218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.348306 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.348312 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.348380 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.348448 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":116}
I20260810 07:19:33.348557 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.344186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:37449 (local address 127.14.164.1:57193)
0810 07:19:33.344396 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.344408 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.344468 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.344616 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.344617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.344619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.344772 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.344774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.345703 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.345703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.347306 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.347312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.348217 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.348218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.348319 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.348321 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.348380 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.348444 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":125,"thread_start_us":130,"threads_started":1}
I20260810 07:19:33.349035 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4f8fbf9fff4693be5ac457bff36d0c" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c" is_pre_election: true
I20260810 07:19:33.349037 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4f8fbf9fff4693be5ac457bff36d0c" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c034d0cdedab4d848c8b66689955d174" is_pre_election: true
I20260810 07:19:33.349224 18431 raft_consensus.cc:2454] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 0.
I20260810 07:19:33.349225 18210 raft_consensus.cc:2454] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 0.
I20260810 07:19:33.349486 17868 leader_election.cc:302] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c034d0cdedab4d848c8b66689955d174, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:33.349658 18551 raft_consensus.cc:2781] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:33.349737 18551 raft_consensus.cc:482] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:33.349763 18551 raft_consensus.cc:3037] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.350528 18551 raft_consensus.cc:504] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.350696 18551 leader_election.cc:288] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 election: Requested vote from peers 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449), c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:33.350886 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4f8fbf9fff4693be5ac457bff36d0c" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c"
I20260810 07:19:33.350953 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4f8fbf9fff4693be5ac457bff36d0c" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c034d0cdedab4d848c8b66689955d174"
I20260810 07:19:33.350989 18210 raft_consensus.cc:3037] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.351034 18431 raft_consensus.cc:3037] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.351979 18210 raft_consensus.cc:2454] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 1.
I20260810 07:19:33.351994 18431 raft_consensus.cc:2454] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 1.
I20260810 07:19:33.352185 17869 leader_election.cc:302] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f30a1d6b52c4426b87f27454efd143c, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:33.352286 18551 raft_consensus.cc:2781] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:33.352478 18551 raft_consensus.cc:686] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 1 LEADER]: Becoming Leader. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Running, Role: LEADER
I20260810 07:19:33.352582 18551 consensus_queue.cc:237] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.353539 17722 catalog_manager.cc:5495] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "cc0e0a7df56e4d4eb4195ef177154727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } health_report { overall_health: UNKNOWN } } }
W20260810 07:19:33.391937 18057 tablet.cc:2369] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 07:19:33 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:33.397002 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.389641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.14.164.62:45167)
0810 07:19:33.389723 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.389726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.389822 (+ 96us) server_negotiation.cc:407] Connection header received
0810 07:19:33.389872 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.389872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.389882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.389939 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.390403 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.390911 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.391583 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.391760 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.391816 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.394310 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.394311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.394312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:33.394318 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.395935 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.396292 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.396293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.396293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.396347 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:33.396518 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:33.396518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:33.396519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:33.396635 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.396686 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.396865 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.396936 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260810 07:19:33.397076 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.389586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45167 (local address 127.0.0.1:48054)
0810 07:19:33.389748 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.389757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.389824 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.389960 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.389961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.390280 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:33.390331 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.390333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.390981 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.390982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.391513 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.391514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.391861 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.391863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.391942 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.392711 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:33.392715 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:33.394219 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:33.396011 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.396018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.396020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.396212 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.396408 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:33.396408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:33.396408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:33.396465 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:33.396689 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:33.396696 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:33.396788 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.396847 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.396989 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":24,"thread_start_us":105,"threads_started":1}
I20260810 07:19:33.399384 17722 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:48054:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:19:33.399642 17722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:33.401314 18399 tablet_service.cc:1467] Processing CreateTablet for tablet 81b76348ce474f769e4d72b0a60f9fe8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=895ef051101342f6a76c49818667093e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:33.401325 18184 tablet_service.cc:1467] Processing CreateTablet for tablet 81b76348ce474f769e4d72b0a60f9fe8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=895ef051101342f6a76c49818667093e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:33.401505 18399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81b76348ce474f769e4d72b0a60f9fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.401453 17957 tablet_service.cc:1467] Processing CreateTablet for tablet 81b76348ce474f769e4d72b0a60f9fe8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=895ef051101342f6a76c49818667093e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:33.401525 18184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81b76348ce474f769e4d72b0a60f9fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.401598 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81b76348ce474f769e4d72b0a60f9fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.402756 18541 tablet_bootstrap.cc:492] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: Bootstrap starting.
I20260810 07:19:33.403137 18543 tablet_bootstrap.cc:492] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: Bootstrap starting.
I20260810 07:19:33.403147 18544 tablet_bootstrap.cc:492] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: Bootstrap starting.
I20260810 07:19:33.403374 18541 tablet_bootstrap.cc:654] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.403630 18544 tablet_bootstrap.cc:654] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.403777 18543 tablet_bootstrap.cc:654] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.404163 18541 tablet_bootstrap.cc:492] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: No bootstrap required, opened a new log
I20260810 07:19:33.404222 18541 ts_tablet_manager.cc:1397] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260810 07:19:33.404349 18544 tablet_bootstrap.cc:492] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: No bootstrap required, opened a new log
I20260810 07:19:33.404410 18544 ts_tablet_manager.cc:1397] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:33.404397 18541 raft_consensus.cc:348] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.404488 18541 raft_consensus.cc:374] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.404511 18541 raft_consensus.cc:729] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.404559 18541 consensus_queue.cc:260] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.404546 18544 raft_consensus.cc:348] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.404618 18544 raft_consensus.cc:374] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.404645 18541 ts_tablet_manager.cc:1428] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.404654 18544 raft_consensus.cc:729] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.404703 18544 consensus_queue.cc:260] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.404834 18544 ts_tablet_manager.cc:1428] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:19:33.404847 18553 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.404950 18553 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.404970 18551 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.405054 18551 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.405579 18543 tablet_bootstrap.cc:492] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: No bootstrap required, opened a new log
I20260810 07:19:33.405637 18543 ts_tablet_manager.cc:1397] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 07:19:33.405795 18543 raft_consensus.cc:348] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.405857 18543 raft_consensus.cc:374] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.405871 18543 raft_consensus.cc:729] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f30a1d6b52c4426b87f27454efd143c, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.405931 18543 consensus_queue.cc:260] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.406023 18543 ts_tablet_manager.cc:1428] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.406096 18554 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.406210 18554 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.470288 18553 raft_consensus.cc:482] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:33.470410 18553 raft_consensus.cc:504] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.470904 18553 leader_election.cc:288] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557), 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449)
I20260810 07:19:33.475596 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.471350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:38659 (local address 127.14.164.1:38557)
0810 07:19:33.471410 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.471412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.471422 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:33.471455 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.471461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.471471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.471517 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.471772 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.472619 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.474492 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.475130 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.475250 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.475259 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.475452 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.475502 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:19:33.475596 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.471292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:51883 (local address 127.14.164.2:37449)
0810 07:19:33.471372 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.471374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.471532 (+ 158us) server_negotiation.cc:407] Connection header received
0810 07:19:33.471638 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.471639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.471647 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.471688 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:33.471876 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.472614 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.474490 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.475143 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.475233 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.475245 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.475452 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.475502 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:19:33.475602 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.471112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38557 (local address 127.14.164.3:38659)
0810 07:19:33.471186 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.471200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.471255 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.471575 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.471577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.471579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.471668 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.471670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.472777 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.472779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.474347 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.474352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.475241 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.475242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.475353 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.475355 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.475419 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.475476 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":143}
I20260810 07:19:33.475612 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.471213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:37449 (local address 127.14.164.3:51883)
0810 07:19:33.471450 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.471461 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.471551 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.471713 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.471714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.471716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:33.471794 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.471796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.472777 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.472779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.474347 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.474352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.475268 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.475269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.475343 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.475348 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.475423 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.475476 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":61,"thread_start_us":115,"threads_started":1}
I20260810 07:19:33.476001 17985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b76348ce474f769e4d72b0a60f9fe8" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e0a7df56e4d4eb4195ef177154727" is_pre_election: true
I20260810 07:19:33.476168 17985 raft_consensus.cc:2454] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 0.
I20260810 07:19:33.476224 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b76348ce474f769e4d72b0a60f9fe8" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c" is_pre_election: true
I20260810 07:19:33.476342 18210 raft_consensus.cc:2454] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 0.
I20260810 07:19:33.476403 18327 leader_election.cc:302] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c034d0cdedab4d848c8b66689955d174, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:33.476588 18553 raft_consensus.cc:2781] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:33.476678 18553 raft_consensus.cc:482] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:33.476701 18553 raft_consensus.cc:3037] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.477562 18553 raft_consensus.cc:504] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.477720 18553 leader_election.cc:288] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 election: Requested vote from peers cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557), 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449)
I20260810 07:19:33.477913 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b76348ce474f769e4d72b0a60f9fe8" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c"
I20260810 07:19:33.477934 17985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b76348ce474f769e4d72b0a60f9fe8" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e0a7df56e4d4eb4195ef177154727"
I20260810 07:19:33.478003 17985 raft_consensus.cc:3037] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.478003 18210 raft_consensus.cc:3037] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.478842 18210 raft_consensus.cc:2454] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 1.
I20260810 07:19:33.478842 17985 raft_consensus.cc:2454] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 1.
I20260810 07:19:33.479059 18327 leader_election.cc:302] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c034d0cdedab4d848c8b66689955d174, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:33.479162 18553 raft_consensus.cc:2781] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:33.479369 18553 raft_consensus.cc:686] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [term 1 LEADER]: Becoming Leader. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Running, Role: LEADER
I20260810 07:19:33.479461 18553 consensus_queue.cc:237] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.479954 18578 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: TxnStatusTablet state changed. Reason: New leader c034d0cdedab4d848c8b66689955d174. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.480089 18578 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:33.480345 17723 catalog_manager.cc:5495] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 reported cstate change: term changed from 0 to 1, leader changed from <none> to c034d0cdedab4d848c8b66689955d174 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } health_report { overall_health: HEALTHY } } }
I20260810 07:19:33.480662 18585 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:19:33.480818 18585 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260810 07:19:33.486901 18510 tablet.cc:2369] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:33.501577 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.498471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.14.164.3:42867)
0810 07:19:33.498577 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.498579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.498614 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:19:33.498719 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.498720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.498744 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.498814 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:33.498974 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.499521 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.500639 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.500928 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.500991 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.501216 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.501325 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.501398 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.501472 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.501492 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 07:19:33.501607 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.498365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:42867 (local address 127.0.0.1:59894)
0810 07:19:33.498558 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.498571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.498631 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.498838 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.498839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.498840 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:33.498896 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.498899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.499603 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.499604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.500533 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.500537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.501005 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.501007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.501076 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.501095 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.501401 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.501410 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.501469 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.501520 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260810 07:19:33.503618 17985 raft_consensus.cc:1261] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Refusing update from remote peer c034d0cdedab4d848c8b66689955d174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:33.503618 18210 raft_consensus.cc:1261] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Refusing update from remote peer c034d0cdedab4d848c8b66689955d174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:33.503934 18578 consensus_queue.cc:1035] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:33.504072 18553 consensus_queue.cc:1035] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:33.505230 18551 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: TxnStatusTablet state changed. Reason: New leader c034d0cdedab4d848c8b66689955d174. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.505337 18551 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.505410 18554 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: TxnStatusTablet state changed. Reason: New leader c034d0cdedab4d848c8b66689955d174. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.505499 18554 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.506318 18578 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.506404 18578 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:33.506930 18578 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.507010 18578 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P c034d0cdedab4d848c8b66689955d174: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:33.507215 18551 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.507304 18551 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P cc0e0a7df56e4d4eb4195ef177154727: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.507515 18589 mvcc.cc:205] Tried to move back new op lower bound from 7316874745867943936 to 7316874745772486656. Current Snapshot: MvccSnapshot[applied={T|T < 7316874745867943936 or (T in {7316874745867943936})}]
I20260810 07:19:33.508589 18554 tablet_replica.cc:439] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } } }
I20260810 07:19:33.508684 18554 tablet_replica.cc:442] T 81b76348ce474f769e4d72b0a60f9fe8 P 0f30a1d6b52c4426b87f27454efd143c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:33.511572 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.508543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.14.164.1:38557)
0810 07:19:33.508588 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.508590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.508599 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:33.508631 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.508632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.508650 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.508747 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:33.508914 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.509623 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.510457 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.510772 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.510846 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.511060 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.511191 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.511276 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.511416 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.511445 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:19:33.517652 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.514792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:59169 (local address 127.14.164.3:42867)
0810 07:19:33.514842 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.514844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.515018 (+ 174us) server_negotiation.cc:407] Connection header received
0810 07:19:33.515048 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.515049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.515061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.515129 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:33.515234 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.515995 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.516835 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.517142 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.517214 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.517404 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.517440 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.517494 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.517547 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.517566 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:19:33.517673 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.514692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:42867 (local address 127.14.164.1:59169)
0810 07:19:33.514925 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.514938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.515006 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.515129 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.515130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.515131 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:33.515191 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.515192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.516099 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.516101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.516699 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.516701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.517224 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.517225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.517289 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.517305 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.517507 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.517509 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.517544 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.517583 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":160,"threads_started":1}
I20260810 07:19:33.520423 18210 raft_consensus.cc:1261] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Refusing update from remote peer cc0e0a7df56e4d4eb4195ef177154727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:33.520458 18431 raft_consensus.cc:1261] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Refusing update from remote peer cc0e0a7df56e4d4eb4195ef177154727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:33.520684 18551 consensus_queue.cc:1035] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:33.520855 18558 consensus_queue.cc:1035] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:33.522082 18591 mvcc.cc:205] Tried to move back new op lower bound from 7316874745937694720 to 7316874745252823040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874745937694720}]
I20260810 07:19:33.531376 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.527822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:41453 (local address 127.14.164.1:38557)
0810 07:19:33.528016 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.528018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.528027 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:33.528050 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.528051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.528059 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.528131 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:33.528321 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.529053 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.530539 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.530821 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.530917 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.531101 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.531140 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.531195 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.531267 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.531292 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260810 07:19:33.531383 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.527790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38557 (local address 127.14.164.3:41453)
0810 07:19:33.527888 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:33.527900 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:33.527940 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:33.528139 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:33.528140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:33.528141 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:33.528220 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.528222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.529177 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.529179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.530191 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:33.530194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.530918 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.530919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:33.530999 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.531013 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:33.531202 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:33.531203 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:19:33.531262 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:33.531312 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 07:19:33.545076 17722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:33.545207 17722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:33.546106 17722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Sending DeleteTablet for 3 replicas of tablet aa4f8fbf9fff4693be5ac457bff36d0c
I20260810 07:19:33.546478 18184 tablet_service.cc:1514] Processing DeleteTablet for tablet aa4f8fbf9fff4693be5ac457bff36d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:33 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38368
I20260810 07:19:33.546535 18399 tablet_service.cc:1514] Processing DeleteTablet for tablet aa4f8fbf9fff4693be5ac457bff36d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:33 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59888
I20260810 07:19:33.546828 18614 tablet_replica.cc:330] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: stopping tablet replica
I20260810 07:19:33.546988 18614 raft_consensus.cc:2229] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:33.547132 18614 raft_consensus.cc:2258] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:33.547060 17722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:33.547217 17722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:33.547672 17957 tablet_service.cc:1514] Processing DeleteTablet for tablet aa4f8fbf9fff4693be5ac457bff36d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:33 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:51568
I20260810 07:19:33.547904 18616 tablet_replica.cc:330] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: stopping tablet replica
I20260810 07:19:33.548015 18616 raft_consensus.cc:2229] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:33.548072 18616 raft_consensus.cc:2258] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:33.548842 17957 tablet_service.cc:1467] Processing CreateTablet for tablet 9879747b58494a8793d59886708bca4b (DEFAULT_TABLE table=test-table [id=3f183f1975e84fe584de62096aa524b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:33.548992 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9879747b58494a8793d59886708bca4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.549016 18614 ts_tablet_manager.cc:1905] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:33.550489 18399 tablet_service.cc:1467] Processing CreateTablet for tablet 9879747b58494a8793d59886708bca4b (DEFAULT_TABLE table=test-table [id=3f183f1975e84fe584de62096aa524b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:33.550638 18399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9879747b58494a8793d59886708bca4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.551173 18614 ts_tablet_manager.cc:1918] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:33.551321 18614 log.cc:1196] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/aa4f8fbf9fff4693be5ac457bff36d0c
I20260810 07:19:33.551738 18614 ts_tablet_manager.cc:1939] T aa4f8fbf9fff4693be5ac457bff36d0c P 0f30a1d6b52c4426b87f27454efd143c: Deleting consensus metadata
I20260810 07:19:33.552002 18184 tablet_service.cc:1467] Processing CreateTablet for tablet 9879747b58494a8793d59886708bca4b (DEFAULT_TABLE table=test-table [id=3f183f1975e84fe584de62096aa524b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:33.552119 18184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9879747b58494a8793d59886708bca4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.553345 18543 tablet_bootstrap.cc:492] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Bootstrap starting.
I20260810 07:19:33.553444 18544 tablet_bootstrap.cc:492] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Bootstrap starting.
I20260810 07:19:33.553768 18543 tablet_bootstrap.cc:654] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.553922 18544 tablet_bootstrap.cc:654] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.554334 18541 tablet_bootstrap.cc:492] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Bootstrap starting.
I20260810 07:19:33.554718 18544 tablet_bootstrap.cc:492] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: No bootstrap required, opened a new log
I20260810 07:19:33.554772 18544 ts_tablet_manager.cc:1397] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:33.554818 18541 tablet_bootstrap.cc:654] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.554924 18544 raft_consensus.cc:348] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.555012 18544 raft_consensus.cc:374] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.555034 18544 raft_consensus.cc:729] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.555078 18544 consensus_queue.cc:260] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.555166 18544 ts_tablet_manager.cc:1428] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.555591 17706 catalog_manager.cc:4841] TS 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449): tablet aa4f8fbf9fff4693be5ac457bff36d0c (table test-table [id=33a2e9b36aa547549595a1a5e6d17f14]) successfully deleted
I20260810 07:19:33.556259 18616 ts_tablet_manager.cc:1905] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:33.557169 18543 tablet_bootstrap.cc:492] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: No bootstrap required, opened a new log
I20260810 07:19:33.557219 18543 ts_tablet_manager.cc:1397] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260810 07:19:33.557374 18543 raft_consensus.cc:348] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.557442 18543 raft_consensus.cc:374] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.557464 18543 raft_consensus.cc:729] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f30a1d6b52c4426b87f27454efd143c, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.557514 18543 consensus_queue.cc:260] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.557605 18543 ts_tablet_manager.cc:1428] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.557763 18617 tablet_replica.cc:330] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: stopping tablet replica
I20260810 07:19:33.558081 18617 raft_consensus.cc:2229] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:33.558326 18617 raft_consensus.cc:2258] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:33.559069 18616 ts_tablet_manager.cc:1918] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:33.559154 18616 log.cc:1196] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/aa4f8fbf9fff4693be5ac457bff36d0c
I20260810 07:19:33.559504 18616 ts_tablet_manager.cc:1939] T aa4f8fbf9fff4693be5ac457bff36d0c P c034d0cdedab4d848c8b66689955d174: Deleting consensus metadata
I20260810 07:19:33.559919 17705 catalog_manager.cc:4841] TS c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867): tablet aa4f8fbf9fff4693be5ac457bff36d0c (table test-table [id=33a2e9b36aa547549595a1a5e6d17f14]) successfully deleted
I20260810 07:19:33.560251 18617 ts_tablet_manager.cc:1905] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:33.562319 18617 ts_tablet_manager.cc:1918] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:33.562399 18617 log.cc:1196] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/aa4f8fbf9fff4693be5ac457bff36d0c
I20260810 07:19:33.562670 18617 ts_tablet_manager.cc:1939] T aa4f8fbf9fff4693be5ac457bff36d0c P cc0e0a7df56e4d4eb4195ef177154727: Deleting consensus metadata
I20260810 07:19:33.563134 17705 catalog_manager.cc:4841] TS cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557): tablet aa4f8fbf9fff4693be5ac457bff36d0c (table test-table [id=33a2e9b36aa547549595a1a5e6d17f14]) successfully deleted
I20260810 07:19:33.563380 18541 tablet_bootstrap.cc:492] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: No bootstrap required, opened a new log
I20260810 07:19:33.563441 18541 ts_tablet_manager.cc:1397] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20260810 07:19:33.563568 18541 raft_consensus.cc:348] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.563659 18541 raft_consensus.cc:374] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.563683 18541 raft_consensus.cc:729] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.563735 18541 consensus_queue.cc:260] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.563848 18541 ts_tablet_manager.cc:1428] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.611796 18558 raft_consensus.cc:482] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:33.611923 18558 raft_consensus.cc:504] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.612134 18558 leader_election.cc:288] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449), c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:33.612454 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9879747b58494a8793d59886708bca4b" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c" is_pre_election: true
I20260810 07:19:33.612565 18210 raft_consensus.cc:2454] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 0.
I20260810 07:19:33.612494 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9879747b58494a8793d59886708bca4b" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c034d0cdedab4d848c8b66689955d174" is_pre_election: true
I20260810 07:19:33.612612 18431 raft_consensus.cc:2454] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 0.
I20260810 07:19:33.612845 17869 leader_election.cc:302] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f30a1d6b52c4426b87f27454efd143c, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:33.612946 18558 raft_consensus.cc:2781] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:33.612972 18558 raft_consensus.cc:482] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:33.612991 18558 raft_consensus.cc:3037] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.613881 18558 raft_consensus.cc:504] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.614028 18558 leader_election.cc:288] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 election: Requested vote from peers 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449), c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:33.614244 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9879747b58494a8793d59886708bca4b" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c034d0cdedab4d848c8b66689955d174"
I20260810 07:19:33.614271 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9879747b58494a8793d59886708bca4b" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c"
I20260810 07:19:33.614321 18431 raft_consensus.cc:3037] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.614351 18210 raft_consensus.cc:3037] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.615681 18210 raft_consensus.cc:2454] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 1.
I20260810 07:19:33.616041 17869 leader_election.cc:302] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f30a1d6b52c4426b87f27454efd143c, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:33.616232 18558 raft_consensus.cc:2781] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:33.616309 18558 raft_consensus.cc:686] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 1 LEADER]: Becoming Leader. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Running, Role: LEADER
I20260810 07:19:33.616300 18431 raft_consensus.cc:2454] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 1.
I20260810 07:19:33.616360 18558 consensus_queue.cc:237] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:33.616962 17723 catalog_manager.cc:5495] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "cc0e0a7df56e4d4eb4195ef177154727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:33.639281 18210 raft_consensus.cc:1261] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Refusing update from remote peer cc0e0a7df56e4d4eb4195ef177154727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:33.639549 18609 consensus_queue.cc:1035] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:33.641039 18431 raft_consensus.cc:1261] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Refusing update from remote peer cc0e0a7df56e4d4eb4195ef177154727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:33.641275 18593 mvcc.cc:205] Tried to move back new op lower bound from 7316874746424979456 to 7316874746332983296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874746424979456}]
I20260810 07:19:33.641640 18558 consensus_queue.cc:1035] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:19:33.646291 18591 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:19:35.166100 17722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:35.166218 17722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:35.167220 17722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Sending DeleteTablet for 3 replicas of tablet 9879747b58494a8793d59886708bca4b
I20260810 07:19:35.167457 17957 tablet_service.cc:1514] Processing DeleteTablet for tablet 9879747b58494a8793d59886708bca4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:35 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:51568
I20260810 07:19:35.167492 18399 tablet_service.cc:1514] Processing DeleteTablet for tablet 9879747b58494a8793d59886708bca4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:35 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59888
I20260810 07:19:35.167658 18184 tablet_service.cc:1514] Processing DeleteTablet for tablet 9879747b58494a8793d59886708bca4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:35 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38368
I20260810 07:19:35.167819 18664 tablet_replica.cc:330] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: stopping tablet replica
I20260810 07:19:35.167905 18665 tablet_replica.cc:330] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: stopping tablet replica
I20260810 07:19:35.167918 18664 raft_consensus.cc:2229] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:35.167976 18665 raft_consensus.cc:2229] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:35.168022 18665 raft_consensus.cc:2258] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:35.168123 18668 tablet_replica.cc:330] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: stopping tablet replica
I20260810 07:19:35.168120 18664 raft_consensus.cc:2258] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:35.168179 18668 raft_consensus.cc:2229] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:35.168217 18668 raft_consensus.cc:2258] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:35.168452 18665 ts_tablet_manager.cc:1905] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:35.168493 18664 ts_tablet_manager.cc:1905] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:35.170311 18668 ts_tablet_manager.cc:1905] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:35.170324 18665 ts_tablet_manager.cc:1918] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:19:35.170460 18665 log.cc:1196] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/9879747b58494a8793d59886708bca4b
I20260810 07:19:35.170756 18665 ts_tablet_manager.cc:1939] T 9879747b58494a8793d59886708bca4b P c034d0cdedab4d848c8b66689955d174: Deleting consensus metadata
I20260810 07:19:35.170926 18664 ts_tablet_manager.cc:1918] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:19:35.171000 18664 log.cc:1196] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/9879747b58494a8793d59886708bca4b
I20260810 07:19:35.171247 18664 ts_tablet_manager.cc:1939] T 9879747b58494a8793d59886708bca4b P cc0e0a7df56e4d4eb4195ef177154727: Deleting consensus metadata
I20260810 07:19:35.171475 17705 catalog_manager.cc:4841] TS c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867): tablet 9879747b58494a8793d59886708bca4b (table test-table [id=3f183f1975e84fe584de62096aa524b8]) successfully deleted
I20260810 07:19:35.171581 17705 catalog_manager.cc:4841] TS cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557): tablet 9879747b58494a8793d59886708bca4b (table test-table [id=3f183f1975e84fe584de62096aa524b8]) successfully deleted
I20260810 07:19:35.172257 18668 ts_tablet_manager.cc:1918] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:19:35.172326 18668 log.cc:1196] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/9879747b58494a8793d59886708bca4b
I20260810 07:19:35.172554 18668 ts_tablet_manager.cc:1939] T 9879747b58494a8793d59886708bca4b P 0f30a1d6b52c4426b87f27454efd143c: Deleting consensus metadata
I20260810 07:19:35.172938 17706 catalog_manager.cc:4841] TS 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449): tablet 9879747b58494a8793d59886708bca4b (table test-table [id=3f183f1975e84fe584de62096aa524b8]) successfully deleted
Aug 10 07:19:35 dist-test-slave-x5fp krb5kdc[17649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346375, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:35 dist-test-slave-x5fp krb5kdc[17649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346375, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:35.194319 18679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.186173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.14.164.62:45167)
0810 07:19:35.186340 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:35.186342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:35.186392 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:19:35.186493 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:35.186494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:35.186508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:35.186603 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:35.187281 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.187865 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.188849 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.189072 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.189147 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:35.191686 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:35.191687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:35.191688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:35.191694 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:35.193137 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:35.193509 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:35.193510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:35.193511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:35.193659 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:35.193847 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:35.193848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:35.193848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:35.193986 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:35.194065 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:35.194175 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 07:19:35.194244 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260810 07:19:35.196316 17722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:35.196477 17722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:35.198179 18184 tablet_service.cc:1467] Processing CreateTablet for tablet 902411a382f945508cf1bf3364cf2b8e (DEFAULT_TABLE table=test-table [id=7884b79a2468404d9d66a496dbd94d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:35.198251 17957 tablet_service.cc:1467] Processing CreateTablet for tablet 902411a382f945508cf1bf3364cf2b8e (DEFAULT_TABLE table=test-table [id=7884b79a2468404d9d66a496dbd94d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:35.198357 18184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902411a382f945508cf1bf3364cf2b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.198377 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902411a382f945508cf1bf3364cf2b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.198374 18399 tablet_service.cc:1467] Processing CreateTablet for tablet 902411a382f945508cf1bf3364cf2b8e (DEFAULT_TABLE table=test-table [id=7884b79a2468404d9d66a496dbd94d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:35.198545 18399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902411a382f945508cf1bf3364cf2b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.199956 18680 tablet_bootstrap.cc:492] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Bootstrap starting.
I20260810 07:19:35.200013 18689 tablet_bootstrap.cc:492] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Bootstrap starting.
I20260810 07:19:35.200489 18689 tablet_bootstrap.cc:654] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.200490 18680 tablet_bootstrap.cc:654] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.200901 18684 tablet_bootstrap.cc:492] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Bootstrap starting.
I20260810 07:19:35.201138 18689 tablet_bootstrap.cc:492] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: No bootstrap required, opened a new log
I20260810 07:19:35.201186 18689 ts_tablet_manager.cc:1397] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:35.201275 18689 raft_consensus.cc:348] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.201336 18689 raft_consensus.cc:374] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.201350 18689 raft_consensus.cc:729] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.201391 18684 tablet_bootstrap.cc:654] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.201423 18689 consensus_queue.cc:260] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.201531 18689 ts_tablet_manager.cc:1428] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:35.201792 18680 tablet_bootstrap.cc:492] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: No bootstrap required, opened a new log
I20260810 07:19:35.201874 18680 ts_tablet_manager.cc:1397] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:19:35.202129 18684 tablet_bootstrap.cc:492] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: No bootstrap required, opened a new log
I20260810 07:19:35.202042 18680 raft_consensus.cc:348] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.202245 18680 raft_consensus.cc:374] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.202282 18680 raft_consensus.cc:729] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.202220 18684 ts_tablet_manager.cc:1397] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:35.202337 18680 consensus_queue.cc:260] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.202462 18680 ts_tablet_manager.cc:1428] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:35.202869 18684 raft_consensus.cc:348] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.202968 18684 raft_consensus.cc:374] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.203003 18684 raft_consensus.cc:729] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f30a1d6b52c4426b87f27454efd143c, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.203055 18684 consensus_queue.cc:260] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.203250 18684 ts_tablet_manager.cc:1428] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:35.292959 18648 raft_consensus.cc:482] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:35.293114 18648 raft_consensus.cc:504] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.293326 18648 leader_election.cc:288] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449), c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:35.293555 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902411a382f945508cf1bf3364cf2b8e" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c034d0cdedab4d848c8b66689955d174" is_pre_election: true
I20260810 07:19:35.293576 18209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902411a382f945508cf1bf3364cf2b8e" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c" is_pre_election: true
I20260810 07:19:35.293686 18431 raft_consensus.cc:2454] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 0.
I20260810 07:19:35.293684 18209 raft_consensus.cc:2454] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 0.
I20260810 07:19:35.293916 17869 leader_election.cc:302] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f30a1d6b52c4426b87f27454efd143c, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:35.294034 18648 raft_consensus.cc:2781] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:35.294086 18648 raft_consensus.cc:482] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:35.294107 18648 raft_consensus.cc:3037] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.294866 18648 raft_consensus.cc:504] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.295066 18648 leader_election.cc:288] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 election: Requested vote from peers 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449), c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867)
I20260810 07:19:35.295244 18209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902411a382f945508cf1bf3364cf2b8e" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c"
I20260810 07:19:35.295342 18209 raft_consensus.cc:3037] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.295296 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902411a382f945508cf1bf3364cf2b8e" candidate_uuid: "cc0e0a7df56e4d4eb4195ef177154727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c034d0cdedab4d848c8b66689955d174"
I20260810 07:19:35.295410 18431 raft_consensus.cc:3037] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.296041 18209 raft_consensus.cc:2454] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 1.
I20260810 07:19:35.296268 17869 leader_election.cc:302] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f30a1d6b52c4426b87f27454efd143c, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:35.296387 18648 raft_consensus.cc:2781] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:35.296378 18431 raft_consensus.cc:2454] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0e0a7df56e4d4eb4195ef177154727 in term 1.
I20260810 07:19:35.296447 18648 raft_consensus.cc:686] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 1 LEADER]: Becoming Leader. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Running, Role: LEADER
I20260810 07:19:35.296504 18648 consensus_queue.cc:237] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.297345 17722 catalog_manager.cc:5495] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "cc0e0a7df56e4d4eb4195ef177154727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:35.324563 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.322123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51574 (local address 127.14.164.1:38557)
0810 07:19:35.322285 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:35.322287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:35.322293 (+ 6us) server_negotiation.cc:407] Connection header received
0810 07:19:35.322314 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:35.322314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:35.322325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:35.322378 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:35.322552 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.323042 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.323812 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.324006 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.324063 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:35.324266 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:35.324303 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:35.324347 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:35.324423 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 07:19:35.324443 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260810 07:19:35.327697 18209 raft_consensus.cc:1261] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Refusing update from remote peer cc0e0a7df56e4d4eb4195ef177154727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:35.327736 18431 raft_consensus.cc:1261] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Refusing update from remote peer cc0e0a7df56e4d4eb4195ef177154727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:35.328008 18652 consensus_queue.cc:1035] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:35.328133 18648 consensus_queue.cc:1035] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:35.338514 17722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:35.338634 17722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:35.339305 17722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Sending DeleteTablet for 3 replicas of tablet 902411a382f945508cf1bf3364cf2b8e
I20260810 07:19:35.339653 18399 tablet_service.cc:1514] Processing DeleteTablet for tablet 902411a382f945508cf1bf3364cf2b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:35 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59888
I20260810 07:19:35.339694 18184 tablet_service.cc:1514] Processing DeleteTablet for tablet 902411a382f945508cf1bf3364cf2b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:35 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38368
I20260810 07:19:35.339776 18665 tablet_replica.cc:330] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: stopping tablet replica
I20260810 07:19:35.339854 18665 raft_consensus.cc:2229] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:35.339859 18668 tablet_replica.cc:330] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: stopping tablet replica
I20260810 07:19:35.339911 18665 raft_consensus.cc:2258] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:35.339941 18668 raft_consensus.cc:2229] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:35.339985 18668 raft_consensus.cc:2258] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:35.340015 17722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:35.340186 17722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:35.340246 17957 tablet_service.cc:1514] Processing DeleteTablet for tablet 902411a382f945508cf1bf3364cf2b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:35 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:51568
I20260810 07:19:35.340306 18665 ts_tablet_manager.cc:1905] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:35.340374 18664 tablet_replica.cc:330] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: stopping tablet replica
I20260810 07:19:35.340443 18664 raft_consensus.cc:2229] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:35.340503 18664 raft_consensus.cc:2258] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:35.341071 18664 ts_tablet_manager.cc:1905] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:35.341655 18184 tablet_service.cc:1467] Processing CreateTablet for tablet 084b4cbdea6e4d60a10e3aa6c6797185 (DEFAULT_TABLE table=test-table [id=16d8a06783254babb674d082d4467efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:35.341784 18184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b4cbdea6e4d60a10e3aa6c6797185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.342451 18668 ts_tablet_manager.cc:1905] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:35.342967 18664 ts_tablet_manager.cc:1918] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:35.343024 18664 log.cc:1196] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/902411a382f945508cf1bf3364cf2b8e
I20260810 07:19:35.343266 18664 ts_tablet_manager.cc:1939] T 902411a382f945508cf1bf3364cf2b8e P cc0e0a7df56e4d4eb4195ef177154727: Deleting consensus metadata
I20260810 07:19:35.343466 18684 tablet_bootstrap.cc:492] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Bootstrap starting.
I20260810 07:19:35.343539 18665 ts_tablet_manager.cc:1918] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:35.343602 18665 log.cc:1196] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/902411a382f945508cf1bf3364cf2b8e
I20260810 07:19:35.343681 18399 tablet_service.cc:1467] Processing CreateTablet for tablet 084b4cbdea6e4d60a10e3aa6c6797185 (DEFAULT_TABLE table=test-table [id=16d8a06783254babb674d082d4467efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:35.343782 18399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b4cbdea6e4d60a10e3aa6c6797185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.343873 18665 ts_tablet_manager.cc:1939] T 902411a382f945508cf1bf3364cf2b8e P c034d0cdedab4d848c8b66689955d174: Deleting consensus metadata
I20260810 07:19:35.343963 18684 tablet_bootstrap.cc:654] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.344019 18668 ts_tablet_manager.cc:1918] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:35.344072 18668 log.cc:1196] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/902411a382f945508cf1bf3364cf2b8e
I20260810 07:19:35.344353 18668 ts_tablet_manager.cc:1939] T 902411a382f945508cf1bf3364cf2b8e P 0f30a1d6b52c4426b87f27454efd143c: Deleting consensus metadata
I20260810 07:19:35.344777 18684 tablet_bootstrap.cc:492] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: No bootstrap required, opened a new log
I20260810 07:19:35.344842 18684 ts_tablet_manager.cc:1397] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:35.345006 18684 raft_consensus.cc:348] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.345110 18684 raft_consensus.cc:374] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.345121 18689 tablet_bootstrap.cc:492] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Bootstrap starting.
I20260810 07:19:35.345139 18684 raft_consensus.cc:729] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f30a1d6b52c4426b87f27454efd143c, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.345185 18684 consensus_queue.cc:260] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.345218 17705 catalog_manager.cc:4841] TS c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867): tablet 902411a382f945508cf1bf3364cf2b8e (table test-table [id=7884b79a2468404d9d66a496dbd94d62]) successfully deleted
I20260810 07:19:35.345293 18684 ts_tablet_manager.cc:1428] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 07:19:35.345533 18689 tablet_bootstrap.cc:654] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.345573 17705 catalog_manager.cc:4841] TS cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557): tablet 902411a382f945508cf1bf3364cf2b8e (table test-table [id=7884b79a2468404d9d66a496dbd94d62]) successfully deleted
I20260810 07:19:35.345788 17957 tablet_service.cc:1467] Processing CreateTablet for tablet 084b4cbdea6e4d60a10e3aa6c6797185 (DEFAULT_TABLE table=test-table [id=16d8a06783254babb674d082d4467efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:35.345824 17706 catalog_manager.cc:4841] TS 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449): tablet 902411a382f945508cf1bf3364cf2b8e (table test-table [id=7884b79a2468404d9d66a496dbd94d62]) successfully deleted
I20260810 07:19:35.345906 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b4cbdea6e4d60a10e3aa6c6797185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.346320 18689 tablet_bootstrap.cc:492] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: No bootstrap required, opened a new log
I20260810 07:19:35.346366 18689 ts_tablet_manager.cc:1397] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:35.346477 18689 raft_consensus.cc:348] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.346526 18689 raft_consensus.cc:374] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.346541 18689 raft_consensus.cc:729] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.346611 18689 consensus_queue.cc:260] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.346711 18689 ts_tablet_manager.cc:1428] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:35.347097 18680 tablet_bootstrap.cc:492] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727: Bootstrap starting.
I20260810 07:19:35.347420 18680 tablet_bootstrap.cc:654] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.348110 18680 tablet_bootstrap.cc:492] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727: No bootstrap required, opened a new log
I20260810 07:19:35.348181 18680 ts_tablet_manager.cc:1397] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:35.348304 18680 raft_consensus.cc:348] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.348387 18680 raft_consensus.cc:374] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.348414 18680 raft_consensus.cc:729] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e0a7df56e4d4eb4195ef177154727, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.348459 18680 consensus_queue.cc:260] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.348564 18680 ts_tablet_manager.cc:1428] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 07:19:35.480185 18553 raft_consensus.cc:482] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:35.480332 18553 raft_consensus.cc:504] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.480613 18553 leader_election.cc:288] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557), 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449)
I20260810 07:19:35.480918 17985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084b4cbdea6e4d60a10e3aa6c6797185" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e0a7df56e4d4eb4195ef177154727" is_pre_election: true
I20260810 07:19:35.480935 18209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084b4cbdea6e4d60a10e3aa6c6797185" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c" is_pre_election: true
I20260810 07:19:35.481045 18209 raft_consensus.cc:2454] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 0.
I20260810 07:19:35.481046 17985 raft_consensus.cc:2454] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 0.
I20260810 07:19:35.481244 18327 leader_election.cc:302] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c034d0cdedab4d848c8b66689955d174, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:35.481376 18553 raft_consensus.cc:2781] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:35.481442 18553 raft_consensus.cc:482] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:35.481475 18553 raft_consensus.cc:3037] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.482221 18553 raft_consensus.cc:504] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.482405 18553 leader_election.cc:288] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 election: Requested vote from peers cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557), 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449)
I20260810 07:19:35.482592 17985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084b4cbdea6e4d60a10e3aa6c6797185" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e0a7df56e4d4eb4195ef177154727"
I20260810 07:19:35.482683 18209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084b4cbdea6e4d60a10e3aa6c6797185" candidate_uuid: "c034d0cdedab4d848c8b66689955d174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f30a1d6b52c4426b87f27454efd143c"
I20260810 07:19:35.482738 17985 raft_consensus.cc:3037] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.482820 18209 raft_consensus.cc:3037] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.483711 17985 raft_consensus.cc:2454] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 1.
I20260810 07:19:35.483706 18209 raft_consensus.cc:2454] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c034d0cdedab4d848c8b66689955d174 in term 1.
I20260810 07:19:35.483987 18327 leader_election.cc:302] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c034d0cdedab4d848c8b66689955d174, cc0e0a7df56e4d4eb4195ef177154727; no voters:
I20260810 07:19:35.484122 18553 raft_consensus.cc:2781] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:35.484187 18553 raft_consensus.cc:686] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 1 LEADER]: Becoming Leader. State: Replica: c034d0cdedab4d848c8b66689955d174, State: Running, Role: LEADER
I20260810 07:19:35.484244 18553 consensus_queue.cc:237] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } }
I20260810 07:19:35.485008 17723 catalog_manager.cc:5495] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 reported cstate change: term changed from 0 to 1, leader changed from <none> to c034d0cdedab4d848c8b66689955d174 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "c034d0cdedab4d848c8b66689955d174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c034d0cdedab4d848c8b66689955d174" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42867 } health_report { overall_health: HEALTHY } } }
I20260810 07:19:35.496310 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.493437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.14.164.3:42867)
0810 07:19:35.493655 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:35.493657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:35.493667 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:35.493689 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:35.493690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:35.493701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:35.493778 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:35.493903 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.494645 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.495377 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.495684 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.495750 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:35.495931 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:35.495967 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:35.496032 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:35.496212 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:19:35.496238 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
I20260810 07:19:35.501004 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.498663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:36037 (local address 127.14.164.3:42867)
0810 07:19:35.498699 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:35.498701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:35.498810 (+ 109us) server_negotiation.cc:407] Connection header received
0810 07:19:35.498856 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:35.498856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:35.498866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:35.498942 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:35.499113 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.499558 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.500297 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.500479 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.500534 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:35.500687 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:35.500711 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:35.500812 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:35.500882 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:19:35.500927 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:19:35.501083 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.498599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:42867 (local address 127.14.164.3:36037)
0810 07:19:35.498733 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:35.498744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:35.498809 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:35.498988 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:35.498989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:35.498991 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:35.499045 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:35.499046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.499626 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.499627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:35.500225 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:35.500227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:35.500566 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:35.500568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:35.500620 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:35.500625 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:35.500815 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:35.500818 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:35.500878 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 07:19:35.500923 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260810 07:19:35.503268 18209 raft_consensus.cc:1261] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Refusing update from remote peer c034d0cdedab4d848c8b66689955d174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:35.503281 17985 raft_consensus.cc:1261] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727 [term 1 FOLLOWER]: Refusing update from remote peer c034d0cdedab4d848c8b66689955d174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:35.503522 18553 consensus_queue.cc:1035] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0e0a7df56e4d4eb4195ef177154727" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:35.503636 18641 consensus_queue.cc:1035] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f30a1d6b52c4426b87f27454efd143c" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:19:35.508543 18627 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:19:37.028576 17728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:37.028712 17728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:37.029709 17728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a38ea3400d473487a092bff0c0072a: Sending DeleteTablet for 3 replicas of tablet 084b4cbdea6e4d60a10e3aa6c6797185
I20260810 07:19:37.030038 18395 tablet_service.cc:1514] Processing DeleteTablet for tablet 084b4cbdea6e4d60a10e3aa6c6797185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59888
I20260810 07:19:37.030027 17957 tablet_service.cc:1514] Processing DeleteTablet for tablet 084b4cbdea6e4d60a10e3aa6c6797185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:51568
I20260810 07:19:37.030241 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17843
I20260810 07:19:37.030278 18184 tablet_service.cc:1514] Processing DeleteTablet for tablet 084b4cbdea6e4d60a10e3aa6c6797185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38368
I20260810 07:19:37.030292 18765 tablet_replica.cc:330] T 084b4cbdea6e4d60a10e3aa6c6797185 P cc0e0a7df56e4d4eb4195ef177154727: stopping tablet replica
I20260810 07:19:37.046427 18764 tablet_replica.cc:330] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: stopping tablet replica
I20260810 07:19:37.046538 18764 raft_consensus.cc:2229] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:37.046671 18764 raft_consensus.cc:2258] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:37.046974 18764 ts_tablet_manager.cc:1905] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:37.048692 18764 ts_tablet_manager.cc:1918] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:19:37.048810 18764 log.cc:1196] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/084b4cbdea6e4d60a10e3aa6c6797185
I20260810 07:19:37.049103 18764 ts_tablet_manager.cc:1939] T 084b4cbdea6e4d60a10e3aa6c6797185 P c034d0cdedab4d848c8b66689955d174: Deleting consensus metadata
I20260810 07:19:37.049252 18766 tablet_replica.cc:330] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: stopping tablet replica
I20260810 07:19:37.049328 18766 raft_consensus.cc:2229] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:37.049417 18766 raft_consensus.cc:2258] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:37.049572 17705 catalog_manager.cc:4841] TS c034d0cdedab4d848c8b66689955d174 (127.14.164.3:42867): tablet 084b4cbdea6e4d60a10e3aa6c6797185 (table test-table [id=16d8a06783254babb674d082d4467efe]) successfully deleted
I20260810 07:19:37.049710 18766 ts_tablet_manager.cc:1905] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:37.050962 18766 ts_tablet_manager.cc:1918] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:19:37.051061 18766 log.cc:1196] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/084b4cbdea6e4d60a10e3aa6c6797185
I20260810 07:19:37.051270 18766 ts_tablet_manager.cc:1939] T 084b4cbdea6e4d60a10e3aa6c6797185 P 0f30a1d6b52c4426b87f27454efd143c: Deleting consensus metadata
I20260810 07:19:37.051648 17706 catalog_manager.cc:4841] TS 0f30a1d6b52c4426b87f27454efd143c (127.14.164.2:37449): tablet 084b4cbdea6e4d60a10e3aa6c6797185 (table test-table [id=16d8a06783254babb674d082d4467efe]) successfully deleted
W20260810 07:19:37.053503 17705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:38557 (error 108)
I20260810 07:19:37.054100 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.053900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38557 (local address 127.0.0.1:51578)
0810 07:19:37.054023 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.054062 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:38557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260810 07:19:37.054183 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18071
W20260810 07:19:37.054260 17705 catalog_manager.cc:4584] TS cc0e0a7df56e4d4eb4195ef177154727 (127.14.164.1:38557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 084b4cbdea6e4d60a10e3aa6c6797185: Network error: Client connection negotiation failed: client connection to 127.14.164.1:38557: connect: Connection refused (error 111)
I20260810 07:19:37.069793 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18296
I20260810 07:19:37.084908 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17677
2026-08-10T07:19:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5959 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:37 dist-test-slave-x5fp krb5kdc[18777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:37 dist-test-slave-x5fp krb5kdc[18777](info): set up 2 sockets
Aug 10 07:19:37 dist-test-slave-x5fp krb5kdc[18777](info): commencing operation
krb5kdc: starting...
W20260810 07:19:38.410615 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:38 dist-test-slave-x5fp krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:38.548367 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:44767
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:44767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:38.635332 18805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:38.635560 18805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:38.635584 18805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:38.636775 18805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:38.636830 18805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:38.636857 18805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:38.636869 18805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:38.636883 18805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:38.638423 18805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:44767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:44767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:38.638634 18805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:38.638828 18805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:38.644423 18805 server_base.cc:1034] running on GCE node
W20260810 07:19:38.644379 18814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:38.644447 18812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:38.644379 18810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:38.645038 18805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:38.645366 18805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:38.646572 18805 hybrid_clock.cc:648] HybridClock initialized: now 1786346378646549 us; error 43 us; skew 500 ppm
Aug 10 07:19:38 dist-test-slave-x5fp krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346378, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:38.649681 18805 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:38.650226 18805 webserver.cc:460] Webserver started at http://127.14.164.62:37555/ using document root <none> and password file <none>
I20260810 07:19:38.650501 18805 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:38.650580 18805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:38.650758 18805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:38.651978 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "816a5bf23c7f451a9e1f557899da8048"
format_stamp: "Formatted at 2026-08-10 07:19:38 on dist-test-slave-x5fp"
server_key: "b6c8248d99f1b4851ddb0dacdfd8c068"
server_key_iv: "38e5a15f12b380033588dc540202e03f"
server_key_version: "encryptionkey@0"
I20260810 07:19:38.652397 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "816a5bf23c7f451a9e1f557899da8048"
format_stamp: "Formatted at 2026-08-10 07:19:38 on dist-test-slave-x5fp"
server_key: "b6c8248d99f1b4851ddb0dacdfd8c068"
server_key_iv: "38e5a15f12b380033588dc540202e03f"
server_key_version: "encryptionkey@0"
I20260810 07:19:38.654098 18805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:38.655102 18823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:38.655355 18805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:38.655491 18805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "816a5bf23c7f451a9e1f557899da8048"
format_stamp: "Formatted at 2026-08-10 07:19:38 on dist-test-slave-x5fp"
server_key: "b6c8248d99f1b4851ddb0dacdfd8c068"
server_key_iv: "38e5a15f12b380033588dc540202e03f"
server_key_version: "encryptionkey@0"
I20260810 07:19:38.655579 18805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:38.673619 18805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:38.676100 18805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:38.676316 18805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:38.681126 18805 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:44767
I20260810 07:19:38.681164 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:44767 every 8 connection(s)
I20260810 07:19:38.681576 18805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:38.682538 18911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:38.684456 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18805
I20260810 07:19:38.684556 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:38.684747 14992 external_mini_cluster.cc:1428] Setting key 9ce20ea7b3db9eaf37f12786f5f2ea42
I20260810 07:19:38.685410 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048: Bootstrap starting.
I20260810 07:19:38.686066 18911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:38.686467 18911 log.cc:824] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:38.687336 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048: No bootstrap required, opened a new log
I20260810 07:19:38.689468 18911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816a5bf23c7f451a9e1f557899da8048" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44767 } }
I20260810 07:19:38.689638 18911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:38.689688 18911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816a5bf23c7f451a9e1f557899da8048, State: Initialized, Role: FOLLOWER
I20260810 07:19:38.689822 18911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816a5bf23c7f451a9e1f557899da8048" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44767 } }
I20260810 07:19:38.689900 18911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:38.689924 18911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:38.689992 18911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:38.690611 18911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816a5bf23c7f451a9e1f557899da8048" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44767 } }
I20260810 07:19:38.690752 18911 leader_election.cc:302] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 816a5bf23c7f451a9e1f557899da8048; no voters:
I20260810 07:19:38.690933 18911 leader_election.cc:288] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:38.691075 18919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:38.691238 18911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:38.691277 18919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [term 1 LEADER]: Becoming Leader. State: Replica: 816a5bf23c7f451a9e1f557899da8048, State: Running, Role: LEADER
I20260810 07:19:38.691473 18919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816a5bf23c7f451a9e1f557899da8048" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44767 } }
I20260810 07:19:38.691867 18921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "816a5bf23c7f451a9e1f557899da8048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816a5bf23c7f451a9e1f557899da8048" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44767 } } }
I20260810 07:19:38.691886 18925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 816a5bf23c7f451a9e1f557899da8048. Latest consensus state: current_term: 1 leader_uuid: "816a5bf23c7f451a9e1f557899da8048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816a5bf23c7f451a9e1f557899da8048" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44767 } } }
I20260810 07:19:38.691982 18921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:38.692003 18925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:38.692338 18939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 07:19:38 dist-test-slave-x5fp krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:38.693595 18939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:38.696013 18939 catalog_manager.cc:1347] Generated new cluster ID: 51fdbbc79c21435091ff9f2bca95c97d
I20260810 07:19:38.696072 18939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:38.698029 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.685352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.14.164.62:44767)
0810 07:19:38.685562 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.685573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.685617 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:19:38.686241 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.686249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.686303 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.686466 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:38.687435 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.688600 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.689668 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.690259 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.690426 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.693349 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:38.693353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:38.693353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:38.693367 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:38.695428 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:38.695870 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:38.695873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:38.695874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:38.696000 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:38.696968 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:38.696969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:38.696970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:38.697306 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:38.697460 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.697637 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.697784 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260810 07:19:38.711381 18939 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:38.712138 18939 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:38.727814 18939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 816a5bf23c7f451a9e1f557899da8048: Generated new TSK 0
I20260810 07:19:38.728055 18939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:38.764854 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44767
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:38.857204 18957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:38.857466 18957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:38.857508 18957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:38.858834 18957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:38.858901 18957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:38.858946 18957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:38.860625 18957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:38.860991 18957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:38.861256 18957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:38.867096 18970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:38.867166 18957 server_base.cc:1034] running on GCE node
W20260810 07:19:38.867125 18972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:38.867096 18967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:38.867592 18957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:38.867861 18957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:38.869084 18957 hybrid_clock.cc:648] HybridClock initialized: now 1786346378869051 us; error 44 us; skew 500 ppm
Aug 10 07:19:38 dist-test-slave-x5fp krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346378, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:38.872548 18957 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:38.873157 18957 webserver.cc:460] Webserver started at http://127.14.164.1:33651/ using document root <none> and password file <none>
I20260810 07:19:38.873488 18957 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:38.873575 18957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:38.873761 18957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:38.875097 18957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "44c74f23151e4bb3a3c8839f1a54a324"
format_stamp: "Formatted at 2026-08-10 07:19:38 on dist-test-slave-x5fp"
server_key: "a58331a98929e47c34b7338d6a5b98a9"
server_key_iv: "6d3a9eaf2d55fefce9aab5123b7d520f"
server_key_version: "encryptionkey@0"
I20260810 07:19:38.875550 18957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "44c74f23151e4bb3a3c8839f1a54a324"
format_stamp: "Formatted at 2026-08-10 07:19:38 on dist-test-slave-x5fp"
server_key: "a58331a98929e47c34b7338d6a5b98a9"
server_key_iv: "6d3a9eaf2d55fefce9aab5123b7d520f"
server_key_version: "encryptionkey@0"
I20260810 07:19:38.877391 18957 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260810 07:19:38.878463 18982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:38.878741 18957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:38.878865 18957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "44c74f23151e4bb3a3c8839f1a54a324"
format_stamp: "Formatted at 2026-08-10 07:19:38 on dist-test-slave-x5fp"
server_key: "a58331a98929e47c34b7338d6a5b98a9"
server_key_iv: "6d3a9eaf2d55fefce9aab5123b7d520f"
server_key_version: "encryptionkey@0"
I20260810 07:19:38.878990 18957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:38.899493 18957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:38.902570 18957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:38.902789 18957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:38.903111 18957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:38.903510 18957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:38.903576 18957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:38.903618 18957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:38.903650 18957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:38.910722 18957 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:37697
I20260810 07:19:38.910773 19166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:37697 every 8 connection(s)
I20260810 07:19:38.911154 18957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:38.911811 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18957
I20260810 07:19:38.911891 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:38.912094 14992 external_mini_cluster.cc:1428] Setting key 8fa91b83a303ce561e9d19a74071b283
Aug 10 07:19:38 dist-test-slave-x5fp krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346378, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
I20260810 07:19:38.922957 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.912083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:42385 (local address 127.14.164.62:44767)
0810 07:19:38.912141 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.912144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.912438 (+ 294us) server_negotiation.cc:407] Connection header received
0810 07:19:38.913081 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.913081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.913092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.913138 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:38.914550 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.915407 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.916578 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.916887 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.916977 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.919685 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:38.919687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:38.919687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:38.919698 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:38.921355 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:38.921815 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:38.921816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:38.921817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:38.921887 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:38.922201 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:38.922202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:38.922203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:38.922340 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:38.922421 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.922724 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.922819 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:19:38.923137 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.912060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44767 (local address 127.14.164.1:42385)
0810 07:19:38.912274 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:38.912329 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:38.912970 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:38.913219 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:38.913224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:38.914113 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:38.914416 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:38.914423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.915515 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.915516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:38.916476 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:38.916479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.916991 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.916992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:38.917122 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.917734 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:38.917743 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:38.919545 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:38.921447 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:38.921455 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:38.921459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:38.921708 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:38.921964 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:38.921965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:38.921965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:38.922130 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:38.922412 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:38.922429 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:38.922579 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 07:19:38.922691 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 07:19:38.922878 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
I20260810 07:19:38.923750 19167 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44767
I20260810 07:19:38.923867 19167 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:38.924090 19167 heartbeater.cc:511] Master 127.14.164.62:44767 requested a full tablet report, sending...
I20260810 07:19:38.924691 18847 ts_manager.cc:194] Registered new tserver with Master: 44c74f23151e4bb3a3c8839f1a54a324 (127.14.164.1:37697)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
I20260810 07:19:38.925626 18847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:42385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:38.944470 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44767
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:39.031715 19177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:39.031987 19177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:39.032028 19177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:39.033317 19177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:39.033401 19177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:39.033445 19177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:39.035004 19177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:39.035310 19177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:39.035573 19177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:39.041261 19185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:39.041265 19189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:39.041265 19187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:39.041656 19177 server_base.cc:1034] running on GCE node
I20260810 07:19:39.041930 19177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:39.042202 19177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:39.043383 19177 hybrid_clock.cc:648] HybridClock initialized: now 1786346379043355 us; error 42 us; skew 500 ppm
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:39.046284 19177 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:39.046799 19177 webserver.cc:460] Webserver started at http://127.14.164.2:38499/ using document root <none> and password file <none>
I20260810 07:19:39.047047 19177 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:39.047120 19177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:39.047276 19177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:39.048475 19177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "5f67e3644909432d95863120e648c149"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-x5fp"
server_key: "e8e6068bc9be3f2c10a1021917b2d46f"
server_key_iv: "de12c531080ebc5792a4b8e9a91628a3"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.048962 19177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "5f67e3644909432d95863120e648c149"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-x5fp"
server_key: "e8e6068bc9be3f2c10a1021917b2d46f"
server_key_iv: "de12c531080ebc5792a4b8e9a91628a3"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.050495 19177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:39.051448 19199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.051707 19177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:39.051849 19177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "5f67e3644909432d95863120e648c149"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-x5fp"
server_key: "e8e6068bc9be3f2c10a1021917b2d46f"
server_key_iv: "de12c531080ebc5792a4b8e9a91628a3"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.051951 19177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:39.061339 19177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:39.063907 19177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:39.064088 19177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:39.064368 19177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:39.064711 19177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:39.064854 19177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.064898 19177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:39.064924 19177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.071195 19177 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:37153
I20260810 07:19:39.071264 19387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:37153 every 8 connection(s)
I20260810 07:19:39.071627 19177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:39.080926 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 19177
I20260810 07:19:39.081066 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:39.081332 14992 external_mini_cluster.cc:1428] Setting key c2cc2ca1e39415063a8b28333d98fe45
I20260810 07:19:39.082053 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.072417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:42789 (local address 127.14.164.62:44767)
0810 07:19:39.072480 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:39.072483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:39.072670 (+ 187us) server_negotiation.cc:407] Connection header received
0810 07:19:39.073242 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:39.073243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:39.073252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:39.073322 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:39.074302 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.074880 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.076028 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.076244 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.076314 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:39.078661 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:39.078663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:39.078663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:39.078670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:39.080187 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:39.080772 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:39.080773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:39.080775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:39.080840 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:39.081214 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:39.081215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:39.081216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:39.081357 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:39.081454 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:39.081852 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 07:19:39.081959 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:19:39.082226 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.072338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44767 (local address 127.14.164.2:42789)
0810 07:19:39.072524 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.072569 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:39.073148 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:39.073392 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:39.073395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:39.073980 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:39.074180 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:39.074185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.075054 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.075055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:39.075924 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:39.075927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.076330 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.076331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:39.076441 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:39.077000 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:39.077009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:39.078560 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:39.080295 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:39.080304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:39.080306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:39.080574 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:39.080928 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:39.080929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:39.080929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:39.081111 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:39.081444 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:39.081452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:39.081646 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 07:19:39.081772 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 07:19:39.081966 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260810 07:19:39.082844 19388 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44767
I20260810 07:19:39.082968 19388 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:39.083158 19388 heartbeater.cc:511] Master 127.14.164.62:44767 requested a full tablet report, sending...
I20260810 07:19:39.083582 18847 ts_manager.cc:194] Registered new tserver with Master: 5f67e3644909432d95863120e648c149 (127.14.164.2:37153)
I20260810 07:19:39.083984 18847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:42789
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:39.112401 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44767
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:39.200984 19396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:39.201201 19396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:39.201226 19396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:39.202390 19396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:39.202425 19396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:39.202452 19396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:39.203822 19396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:39.204018 19396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:39.204201 19396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:39.209671 19402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:39.209658 19406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:39.209632 19403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:39.209990 19396 server_base.cc:1034] running on GCE node
I20260810 07:19:39.210224 19396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:39.210559 19396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:39.211737 19396 hybrid_clock.cc:648] HybridClock initialized: now 1786346379211730 us; error 38 us; skew 500 ppm
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:39.214653 19396 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:39.215422 19396 webserver.cc:460] Webserver started at http://127.14.164.3:43329/ using document root <none> and password file <none>
I20260810 07:19:39.215711 19396 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:39.215795 19396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:39.216005 19396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:39.217373 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "2411283bb67d457d87071e75f75deb8c"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-x5fp"
server_key: "50f20da1e466f1db150443f819a09249"
server_key_iv: "75dbad4395c927dfc0ba64ae81db514e"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.217803 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "2411283bb67d457d87071e75f75deb8c"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-x5fp"
server_key: "50f20da1e466f1db150443f819a09249"
server_key_iv: "75dbad4395c927dfc0ba64ae81db514e"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.219336 19396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:39.220228 19415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.220445 19396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:39.220567 19396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "2411283bb67d457d87071e75f75deb8c"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-x5fp"
server_key: "50f20da1e466f1db150443f819a09249"
server_key_iv: "75dbad4395c927dfc0ba64ae81db514e"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.220650 19396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:39.231478 19396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:39.233719 19396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:39.233844 19396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:39.234063 19396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:39.234341 19396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:39.234375 19396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.234400 19396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:39.234414 19396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.241246 19396 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:38589
I20260810 07:19:39.241307 19608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:38589 every 8 connection(s)
I20260810 07:19:39.241678 19396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:39.249248 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 19396
I20260810 07:19:39.249348 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:39.249562 14992 external_mini_cluster.cc:1428] Setting key 7ad8278bce4cdbf13f2e69d2338ab863
I20260810 07:19:39.251910 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.242521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:40317 (local address 127.14.164.62:44767)
0810 07:19:39.242607 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:39.242609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:39.242779 (+ 170us) server_negotiation.cc:407] Connection header received
0810 07:19:39.243348 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:39.243349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:39.243359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:39.243414 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:39.244454 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.245074 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.246148 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.246334 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.246399 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:39.248686 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:39.248687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:39.248688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:39.248696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:39.250465 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:39.250905 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:39.250906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:39.250906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:39.250966 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:39.251267 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:39.251267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:39.251268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:39.251379 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:39.251434 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:39.251746 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 07:19:39.251811 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:19:39.252106 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.242469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44767 (local address 127.14.164.3:40317)
0810 07:19:39.242643 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.242685 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:39.243251 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:39.243480 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:39.243484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:39.244132 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:39.244350 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:39.244355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.245167 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.245168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:39.246062 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:39.246064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.246458 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.246460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:39.246575 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:39.247099 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:39.247108 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:39.248577 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:39.250596 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:39.250610 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:39.250612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:39.250840 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:39.251061 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:39.251062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:39.251062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:39.251192 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:39.251466 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:39.251473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:39.251606 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 07:19:39.251725 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:19:39.251871 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260810 07:19:39.252624 19610 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44767
I20260810 07:19:39.252764 19610 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:39.252997 19610 heartbeater.cc:511] Master 127.14.164.62:44767 requested a full tablet report, sending...
I20260810 07:19:39.253445 18847 ts_manager.cc:194] Registered new tserver with Master: 2411283bb67d457d87071e75f75deb8c (127.14.164.3:38589)
I20260810 07:19:39.253830 18847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:40317
I20260810 07:19:39.262012 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:19:39.277520 18917 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:39.276422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.14.164.62:44767)
0810 07:19:39.276532 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:39.276534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:39.276592 (+ 58us) server_negotiation.cc:407] Connection header received
0810 07:19:39.276644 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:39.276645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:39.276655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:39.276710 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:39.277412 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58256: BlockingRecv error: recv got EOF from 127.0.0.1:58256 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:19:39.278555 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18957
I20260810 07:19:39.296518 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 19177
I20260810 07:19:39.313767 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 19396
I20260810 07:19:39.327996 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18805
2026-08-10T07:19:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2246 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[19629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[19629](info): set up 2 sockets
Aug 10 07:19:39 dist-test-slave-x5fp krb5kdc[19629](info): commencing operation
krb5kdc: starting...
W20260810 07:19:40.652616 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:40 dist-test-slave-x5fp krb5kdc[19629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:40.792930 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:38539
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:38539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:40.880082 19648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:40.880400 19648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:40.880450 19648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:40.881757 19648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:40.881845 19648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:40.881870 19648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:40.881891 19648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:40.881918 19648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:40.883678 19648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:38539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:38539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:40.884016 19648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:40.884290 19648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:40.889678 19648 server_base.cc:1034] running on GCE node
W20260810 07:19:40.889575 19657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:40.889642 19659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:40.889575 19662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:40.890168 19648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:40.890529 19648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:40.891767 19648 hybrid_clock.cc:648] HybridClock initialized: now 1786346380891724 us; error 46 us; skew 500 ppm
Aug 10 07:19:40 dist-test-slave-x5fp krb5kdc[19629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:40.895030 19648 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:40.895535 19648 webserver.cc:460] Webserver started at http://127.14.164.62:43771/ using document root <none> and password file <none>
I20260810 07:19:40.895802 19648 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:40.895881 19648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:40.896049 19648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:40.897333 19648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "75b304f8b05b4d56ac1c50c1e254146b"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-x5fp"
server_key: "b8163047dd1047dbed4502bd1d8caf50"
server_key_iv: "926eb7e64be09ad23959e314a0286b44"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.897744 19648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "75b304f8b05b4d56ac1c50c1e254146b"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-x5fp"
server_key: "b8163047dd1047dbed4502bd1d8caf50"
server_key_iv: "926eb7e64be09ad23959e314a0286b44"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.899361 19648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:19:40.900311 19669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:40.900549 19648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:40.900676 19648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "75b304f8b05b4d56ac1c50c1e254146b"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-x5fp"
server_key: "b8163047dd1047dbed4502bd1d8caf50"
server_key_iv: "926eb7e64be09ad23959e314a0286b44"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.900792 19648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:40.925464 19648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:40.927757 19648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:40.927939 19648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:40.932318 19648 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:38539
I20260810 07:19:40.932374 19762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:38539 every 8 connection(s)
I20260810 07:19:40.933001 19648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:40.933732 19763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:40.936134 19763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b: Bootstrap starting.
I20260810 07:19:40.936705 19763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:40.937060 19763 log.cc:824] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:40.937758 19763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b: No bootstrap required, opened a new log
I20260810 07:19:40.939340 19763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b304f8b05b4d56ac1c50c1e254146b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 38539 } }
I20260810 07:19:40.939489 19763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:40.939512 19763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b304f8b05b4d56ac1c50c1e254146b, State: Initialized, Role: FOLLOWER
I20260810 07:19:40.939662 19763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b304f8b05b4d56ac1c50c1e254146b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 38539 } }
I20260810 07:19:40.939754 19763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:40.939779 19763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:40.939808 19763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:40.939998 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 19648
I20260810 07:19:40.940114 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:40.940326 14992 external_mini_cluster.cc:1428] Setting key 923c1a6df73a6df1c76f289737a6857a
I20260810 07:19:40.941088 19763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b304f8b05b4d56ac1c50c1e254146b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 38539 } }
I20260810 07:19:40.941203 19763 leader_election.cc:302] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b304f8b05b4d56ac1c50c1e254146b; no voters:
I20260810 07:19:40.941401 19763 leader_election.cc:288] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:40.941445 19770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:40.941620 19770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [term 1 LEADER]: Becoming Leader. State: Replica: 75b304f8b05b4d56ac1c50c1e254146b, State: Running, Role: LEADER
I20260810 07:19:40.941684 19763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:40.941706 19770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b304f8b05b4d56ac1c50c1e254146b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 38539 } }
I20260810 07:19:40.942068 19770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b304f8b05b4d56ac1c50c1e254146b. Latest consensus state: current_term: 1 leader_uuid: "75b304f8b05b4d56ac1c50c1e254146b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b304f8b05b4d56ac1c50c1e254146b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 38539 } } }
I20260810 07:19:40.942085 19774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b304f8b05b4d56ac1c50c1e254146b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b304f8b05b4d56ac1c50c1e254146b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 38539 } } }
I20260810 07:19:40.942178 19770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:40.942214 19774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:40.943656 19805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:40.943722 19805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:40.943791 19787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:40.943969 19787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:40.945789 19787 catalog_manager.cc:1347] Generated new cluster ID: f35c1e4859c5487a926f1695d573724d
I20260810 07:19:40.945855 19787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:19:40 dist-test-slave-x5fp krb5kdc[19629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:40.953231 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.940870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.14.164.62:38539)
0810 07:19:40.941074 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.941082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.941110 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:19:40.941253 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.941259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.941328 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.941491 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.942675 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.943976 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.944889 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.945484 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.945604 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.948639 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.948643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.948643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.948657 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.951393 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.951853 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.951856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.951856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.951990 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.952219 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.952220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.952220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.952539 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.952705 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.952860 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.953006 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260810 07:19:40.954337 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:40.970297 19787 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:40.970901 19787 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:40.976085 19787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b304f8b05b4d56ac1c50c1e254146b: Generated new TSK 0
I20260810 07:19:40.976342 19787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:40.979903 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.972263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.14.164.62:38539)
0810 07:19:40.972305 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.972307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.975773 (+ 3466us) server_negotiation.cc:407] Connection header received
0810 07:19:40.975860 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.975861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.975873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.975936 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.976458 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.977310 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.978435 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.978647 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.978708 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.979452 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.979453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.979453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:40.979640 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.979657 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:40.979663 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.979711 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.979801 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.979823 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:40.982575 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 19648
2026-08-10T07:19:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1653 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:41 dist-test-slave-x5fp krb5kdc[19825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:41 dist-test-slave-x5fp krb5kdc[19825](info): set up 2 sockets
Aug 10 07:19:41 dist-test-slave-x5fp krb5kdc[19825](info): commencing operation
krb5kdc: starting...
W20260810 07:19:42.249899 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:42.384860 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39139
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:42.471414 19843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:42.471640 19843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:42.471666 19843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:42.472934 19843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:42.472987 19843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:42.473003 19843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:42.473016 19843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:42.473028 19843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:42.474658 19843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:42.474957 19843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:42.475229 19843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:42.480530 19848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:42.480531 19851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:42.480531 19849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:42.480777 19843 server_base.cc:1034] running on GCE node
I20260810 07:19:42.481233 19843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:42.481561 19843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:42.482738 19843 hybrid_clock.cc:648] HybridClock initialized: now 1786346382482730 us; error 39 us; skew 500 ppm
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:42.491683 19843 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:42.492240 19843 webserver.cc:460] Webserver started at http://127.14.164.62:34975/ using document root <none> and password file <none>
I20260810 07:19:42.492498 19843 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:42.492544 19843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:42.492647 19843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:42.493882 19843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "fb033a65a26d44c1ae396a2e02621805"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "ff3c5ce59461b26119a96ce28469f29e"
server_key_iv: "e17b1b2053de96d7cdc48f773c1002f6"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.494246 19843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "fb033a65a26d44c1ae396a2e02621805"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "ff3c5ce59461b26119a96ce28469f29e"
server_key_iv: "e17b1b2053de96d7cdc48f773c1002f6"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.495749 19843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:42.496693 19863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.497144 19843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:42.497248 19843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "fb033a65a26d44c1ae396a2e02621805"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "ff3c5ce59461b26119a96ce28469f29e"
server_key_iv: "e17b1b2053de96d7cdc48f773c1002f6"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.497327 19843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:42.522212 19843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:42.525220 19843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:42.525372 19843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:42.530653 19843 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:39139
I20260810 07:19:42.530725 19949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:39139 every 8 connection(s)
I20260810 07:19:42.531090 19843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:42.531883 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 19843
I20260810 07:19:42.531975 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:42.532158 14992 external_mini_cluster.cc:1428] Setting key d51676cfbe4b984b338346c8ae43d8b4
I20260810 07:19:42.532476 19950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:42.536084 19950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805: Bootstrap starting.
I20260810 07:19:42.536890 19950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:42.537215 19950 log.cc:824] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:42.538110 19950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805: No bootstrap required, opened a new log
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:42.540518 19950 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb033a65a26d44c1ae396a2e02621805" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39139 } }
I20260810 07:19:42.540695 19950 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:42.540772 19950 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb033a65a26d44c1ae396a2e02621805, State: Initialized, Role: FOLLOWER
I20260810 07:19:42.540966 19950 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb033a65a26d44c1ae396a2e02621805" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39139 } }
I20260810 07:19:42.541090 19950 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:42.541133 19950 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:42.541193 19950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:42.541978 19950 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb033a65a26d44c1ae396a2e02621805" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39139 } }
I20260810 07:19:42.542097 19950 leader_election.cc:302] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb033a65a26d44c1ae396a2e02621805; no voters:
I20260810 07:19:42.542289 19950 leader_election.cc:288] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:42.542392 19961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:42.542578 19961 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [term 1 LEADER]: Becoming Leader. State: Replica: fb033a65a26d44c1ae396a2e02621805, State: Running, Role: LEADER
I20260810 07:19:42.542644 19950 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:42.542685 19961 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb033a65a26d44c1ae396a2e02621805" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39139 } }
I20260810 07:19:42.543118 19963 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb033a65a26d44c1ae396a2e02621805. Latest consensus state: current_term: 1 leader_uuid: "fb033a65a26d44c1ae396a2e02621805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb033a65a26d44c1ae396a2e02621805" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39139 } } }
I20260810 07:19:42.543212 19965 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb033a65a26d44c1ae396a2e02621805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb033a65a26d44c1ae396a2e02621805" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39139 } } }
I20260810 07:19:42.543226 19963 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:42.543447 19965 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:42.543686 19977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:42.544250 19977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:42.545118 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:42.532820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.14.164.62:39139)
0810 07:19:42.533001 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:42.533017 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:42.533043 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:19:42.533701 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:42.533709 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:42.533792 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:42.533921 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:42.534821 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.535744 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.536845 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.537381 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.537513 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:42.540229 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:42.540234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:42.540234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:42.540249 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:42.542685 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:42.543133 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:42.543136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:42.543137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:42.543258 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:42.543499 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:42.543500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:42.543500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:42.543879 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:42.544051 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:42.544679 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 07:19:42.544864 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260810 07:19:42.546348 19977 catalog_manager.cc:1347] Generated new cluster ID: 7187f374e9ef46b89fdc3e07025d104f
I20260810 07:19:42.546398 19977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:42.562548 19977 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:42.563213 19977 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:42.569521 19977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb033a65a26d44c1ae396a2e02621805: Generated new TSK 0
I20260810 07:19:42.569746 19977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:42.611768 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39139
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:42.722004 19996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:42.722226 19996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:42.722252 19996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:42.723444 19996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:42.723488 19996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:42.723516 19996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:42.724984 19996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:42.725176 19996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:42.725368 19996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:42.730618 20010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:42.730683 20008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:42.730692 19996 server_base.cc:1034] running on GCE node
W20260810 07:19:42.730636 20007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:42.731158 19996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:42.731384 19996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:42.732544 19996 hybrid_clock.cc:648] HybridClock initialized: now 1786346382732532 us; error 30 us; skew 500 ppm
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:42.735399 19996 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:42.735877 19996 webserver.cc:460] Webserver started at http://127.14.164.1:34547/ using document root <none> and password file <none>
I20260810 07:19:42.736083 19996 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:42.736128 19996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:42.736230 19996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:42.737452 19996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "d2ced2d6e5c54604ba305f22c5f34ae1"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "c18f74e38ed3cd0fc32bd65de00fadd3"
server_key_iv: "e060712fff161ebca050bcbd4fd63237"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.737813 19996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "d2ced2d6e5c54604ba305f22c5f34ae1"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "c18f74e38ed3cd0fc32bd65de00fadd3"
server_key_iv: "e060712fff161ebca050bcbd4fd63237"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.739291 19996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:42.740187 20021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.740471 19996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:42.740612 19996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "d2ced2d6e5c54604ba305f22c5f34ae1"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "c18f74e38ed3cd0fc32bd65de00fadd3"
server_key_iv: "e060712fff161ebca050bcbd4fd63237"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.740700 19996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:42.755731 19996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:42.758277 19996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:42.758460 19996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:42.758709 19996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:42.759042 19996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:42.759094 19996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.759137 19996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:42.759163 19996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.765162 19996 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:32825
I20260810 07:19:42.765213 20201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:32825 every 8 connection(s)
I20260810 07:19:42.765581 19996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:42.770509 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 19996
I20260810 07:19:42.770654 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:42.770895 14992 external_mini_cluster.cc:1428] Setting key eba55ec9a4f9e725e901fc77ca2587f9
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:42.776484 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:42.766378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37881 (local address 127.14.164.62:39139)
0810 07:19:42.766450 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:42.766452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:42.766673 (+ 221us) server_negotiation.cc:407] Connection header received
0810 07:19:42.767283 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:42.767284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:42.767296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:42.767359 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:42.768372 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.768937 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.769949 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.770162 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.770242 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:42.773073 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:42.773074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:42.773075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:42.773081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:42.774839 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:42.775276 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:42.775277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:42.775278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:42.775348 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:42.775620 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:42.775621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:42.775621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:42.775746 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:42.775822 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:42.776213 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 07:19:42.776325 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:42.776559 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:42.766346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39139 (local address 127.14.164.1:37881)
0810 07:19:42.766542 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:42.766578 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:42.767163 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:42.767448 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:42.767451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:42.768042 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:42.768238 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:42.768242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.769032 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.769033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:42.769851 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:42.769853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.770252 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.770253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:42.770390 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:42.771030 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:42.771040 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:42.772970 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:42.774930 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:42.774938 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:42.774940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:42.775171 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:42.775423 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:42.775424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:42.775425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:42.775538 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:42.775824 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:42.775833 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:42.776038 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 07:19:42.776160 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:19:42.776331 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260810 07:19:42.777336 20202 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39139
I20260810 07:19:42.777444 20202 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:42.777663 20202 heartbeater.cc:511] Master 127.14.164.62:39139 requested a full tablet report, sending...
I20260810 07:19:42.778189 19892 ts_manager.cc:194] Registered new tserver with Master: d2ced2d6e5c54604ba305f22c5f34ae1 (127.14.164.1:32825)
I20260810 07:19:42.778825 19892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:37881
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:42.800388 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39139
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:42.888164 20208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:42.888453 20208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:42.888502 20208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:42.889906 20208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:42.889997 20208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:42.890050 20208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:42.891778 20208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:42.892110 20208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:42.892410 20208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:42.898308 20217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:42.898330 20214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:42.898352 20215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:42.898363 20208 server_base.cc:1034] running on GCE node
I20260810 07:19:42.898775 20208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:42.898995 20208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:42.900183 20208 hybrid_clock.cc:648] HybridClock initialized: now 1786346382900163 us; error 46 us; skew 500 ppm
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:42.903015 20208 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:42.903475 20208 webserver.cc:460] Webserver started at http://127.14.164.2:46581/ using document root <none> and password file <none>
I20260810 07:19:42.903725 20208 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:42.903808 20208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:42.903923 20208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:42.905179 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "5c53d061905446f8b55e592193ca0405"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "29973137aa3835e29af67c77c50346ba"
server_key_iv: "69ba021fb61272b33abf79ce5861c1bb"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.905531 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "5c53d061905446f8b55e592193ca0405"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "29973137aa3835e29af67c77c50346ba"
server_key_iv: "69ba021fb61272b33abf79ce5861c1bb"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.907056 20208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:42.907887 20226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.908125 20208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:42.908219 20208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "5c53d061905446f8b55e592193ca0405"
format_stamp: "Formatted at 2026-08-10 07:19:42 on dist-test-slave-x5fp"
server_key: "29973137aa3835e29af67c77c50346ba"
server_key_iv: "69ba021fb61272b33abf79ce5861c1bb"
server_key_version: "encryptionkey@0"
I20260810 07:19:42.908305 20208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:42.923942 20208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:42.926350 20208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:42.926496 20208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:42.926753 20208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:42.927109 20208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:42.927146 20208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.927171 20208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:42.927219 20208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:42.933116 20208 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:44177
I20260810 07:19:42.933202 20421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:44177 every 8 connection(s)
I20260810 07:19:42.933547 20208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:42.936975 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 20208
I20260810 07:19:42.937090 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:42.937299 14992 external_mini_cluster.cc:1428] Setting key 03bd1b1d80121fc8b0dc565def296c90
Aug 10 07:19:42 dist-test-slave-x5fp krb5kdc[19825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346382, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:42.945181 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:42.934361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:53677 (local address 127.14.164.62:39139)
0810 07:19:42.934431 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:42.934433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:42.934633 (+ 200us) server_negotiation.cc:407] Connection header received
0810 07:19:42.935163 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:42.935164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:42.935175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:42.935239 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:42.936205 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.936776 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.938228 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.938426 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.938482 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:42.941425 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:42.941427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:42.941427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:42.941436 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:42.943067 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:42.943821 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:42.943822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:42.943822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:42.943889 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:42.944290 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:42.944290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:42.944291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:42.944437 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:42.944495 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:42.945033 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 07:19:42.945110 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:19:42.945624 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:42.934319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39139 (local address 127.14.164.2:53677)
0810 07:19:42.934496 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:42.934540 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:42.935071 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:42.935294 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:42.935297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:42.935885 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:42.936091 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:42.936096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.936876 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.936878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:42.938136 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:42.938140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:42.938517 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:42.938518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:42.938676 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:42.939248 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:42.939260 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:42.941311 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:42.943163 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:42.943174 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:42.943177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:42.943702 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:42.944022 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:42.944023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:42.944024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:42.944201 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:42.944519 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:42.944528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:42.944858 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 07:19:42.945015 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:19:42.945258 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260810 07:19:42.946198 20422 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39139
I20260810 07:19:42.946333 20422 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:42.946574 20422 heartbeater.cc:511] Master 127.14.164.62:39139 requested a full tablet report, sending...
I20260810 07:19:42.947031 19892 ts_manager.cc:194] Registered new tserver with Master: 5c53d061905446f8b55e592193ca0405 (127.14.164.2:44177)
I20260810 07:19:42.947425 19892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:53677
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:42.969130 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39139
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:43.060626 20429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:43.061002 20429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:43.061048 20429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:43.062245 20429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:43.062317 20429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:43.062362 20429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:43.063921 20429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:43.064210 20429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:43.064473 20429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:43.070041 20440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:43.070029 20449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:43.070030 20443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:43.070339 20429 server_base.cc:1034] running on GCE node
I20260810 07:19:43.070572 20429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:43.070852 20429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:43.072050 20429 hybrid_clock.cc:648] HybridClock initialized: now 1786346383072029 us; error 29 us; skew 500 ppm
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[19825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346383, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:43.074818 20429 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:43.075389 20429 webserver.cc:460] Webserver started at http://127.14.164.3:36187/ using document root <none> and password file <none>
I20260810 07:19:43.075677 20429 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:43.075767 20429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:43.075947 20429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:43.077237 20429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "3c8291f520aa404a90126f66856f422e"
format_stamp: "Formatted at 2026-08-10 07:19:43 on dist-test-slave-x5fp"
server_key: "b536a4037858e7751cdb75750f21ab75"
server_key_iv: "b6ef1dbd479e40be50724877f0e43127"
server_key_version: "encryptionkey@0"
I20260810 07:19:43.077663 20429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "3c8291f520aa404a90126f66856f422e"
format_stamp: "Formatted at 2026-08-10 07:19:43 on dist-test-slave-x5fp"
server_key: "b536a4037858e7751cdb75750f21ab75"
server_key_iv: "b6ef1dbd479e40be50724877f0e43127"
server_key_version: "encryptionkey@0"
I20260810 07:19:43.079205 20429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:19:43.080176 20462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:43.080428 20429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:43.080569 20429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "3c8291f520aa404a90126f66856f422e"
format_stamp: "Formatted at 2026-08-10 07:19:43 on dist-test-slave-x5fp"
server_key: "b536a4037858e7751cdb75750f21ab75"
server_key_iv: "b6ef1dbd479e40be50724877f0e43127"
server_key_version: "encryptionkey@0"
I20260810 07:19:43.080658 20429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:43.103818 20429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:43.106469 20429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:43.106658 20429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:43.106966 20429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:43.107345 20429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:43.107415 20429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:43.107463 20429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:43.107494 20429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:43.114337 20429 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:35805
I20260810 07:19:43.114394 20664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:35805 every 8 connection(s)
I20260810 07:19:43.114753 20429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:43.116756 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 20429
I20260810 07:19:43.116858 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:43.117065 14992 external_mini_cluster.cc:1428] Setting key 9f1c8e295272cd5f36f15f5f250b815f
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[19825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346383, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:43.125541 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:43.115582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:36611 (local address 127.14.164.62:39139)
0810 07:19:43.115629 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:43.115631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:43.115870 (+ 239us) server_negotiation.cc:407] Connection header received
0810 07:19:43.116423 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:43.116425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:43.116438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:43.116551 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:43.117798 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:43.118361 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:43.119483 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:43.119722 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:43.119812 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:43.122397 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:43.122398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:43.122398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:43.122406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:43.123966 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:43.124433 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:43.124434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:43.124435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:43.124516 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:43.124863 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:43.124864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:43.124865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:43.125005 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:43.125069 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:43.125386 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 07:19:43.125466 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:19:43.125666 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:43.115513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39139 (local address 127.14.164.3:36611)
0810 07:19:43.115695 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:43.115743 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:43.116288 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:43.116623 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:43.116626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:43.117449 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:43.117683 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:43.117689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:43.118473 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:43.118475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:43.119379 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:43.119381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:43.119821 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:43.119822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:43.119972 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:43.120566 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:43.120575 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:43.122266 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:43.124051 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:43.124057 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:43.124059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:43.124299 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:43.124603 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:43.124604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:43.124604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:43.124746 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:43.125087 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:43.125094 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:43.125227 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 07:19:43.125324 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:19:43.125457 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260810 07:19:43.126241 20665 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39139
I20260810 07:19:43.126338 20665 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:43.126566 20665 heartbeater.cc:511] Master 127.14.164.62:39139 requested a full tablet report, sending...
I20260810 07:19:43.127053 19892 ts_manager.cc:194] Registered new tserver with Master: 3c8291f520aa404a90126f66856f422e (127.14.164.3:35805)
I20260810 07:19:43.127455 19892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:36611
I20260810 07:19:43.129161 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[19825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346383, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[19825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346383, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:43.156184 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:43.147421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.14.164.62:39139)
0810 07:19:43.147503 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:43.147506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:43.147664 (+ 158us) server_negotiation.cc:407] Connection header received
0810 07:19:43.147727 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:43.147729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:43.147742 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:43.147812 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:43.148407 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:43.149065 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:43.150001 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:43.150212 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:43.150283 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:43.152831 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:43.152833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:43.152834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:43.152843 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:43.154654 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:43.155177 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:43.155178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:43.155178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:43.155254 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:43.155470 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:43.155471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:43.155471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:43.155651 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:43.155789 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:43.155956 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 07:19:43.156045 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 07:19:43.156543 19892 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54980
I20260810 07:19:43.157153 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 19996
I20260810 07:19:43.175642 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 20208
I20260810 07:19:43.188861 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 20429
I20260810 07:19:43.203482 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 19843
2026-08-10T07:19:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2214 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[20681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[20681](info): set up 2 sockets
Aug 10 07:19:43 dist-test-slave-x5fp krb5kdc[20681](info): commencing operation
krb5kdc: starting...
W20260810 07:19:44.463625 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:44 dist-test-slave-x5fp krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:44.604362 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:33677
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:33677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:44.693171 20702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:44.693410 20702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:44.693436 20702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:44.694717 20702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:44.694775 20702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:44.694790 20702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:44.694803 20702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:44.694816 20702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:44.696538 20702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:33677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:33677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:44.696863 20702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:44.697132 20702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:44.702592 20726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:44.702693 20702 server_base.cc:1034] running on GCE node
W20260810 07:19:44.702631 20720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:44.702631 20716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:44.703099 20702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:44.703362 20702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:44.704604 20702 hybrid_clock.cc:648] HybridClock initialized: now 1786346384704581 us; error 40 us; skew 500 ppm
Aug 10 07:19:44 dist-test-slave-x5fp krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346384, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:44.707543 20702 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:44.708072 20702 webserver.cc:460] Webserver started at http://127.14.164.62:33303/ using document root <none> and password file <none>
I20260810 07:19:44.708343 20702 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:44.708420 20702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:44.708575 20702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:44.709797 20702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "fefb6d001852490faddb3018a8233c11"
format_stamp: "Formatted at 2026-08-10 07:19:44 on dist-test-slave-x5fp"
server_key: "71efd15b3cbd46e2b8eecd44110225ee"
server_key_iv: "ce1b2ef27c4afa5318f9e57dfaddc38a"
server_key_version: "encryptionkey@0"
I20260810 07:19:44.710193 20702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "fefb6d001852490faddb3018a8233c11"
format_stamp: "Formatted at 2026-08-10 07:19:44 on dist-test-slave-x5fp"
server_key: "71efd15b3cbd46e2b8eecd44110225ee"
server_key_iv: "ce1b2ef27c4afa5318f9e57dfaddc38a"
server_key_version: "encryptionkey@0"
I20260810 07:19:44.711691 20702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:44.712628 20738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:44.712870 20702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:44.712998 20702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "fefb6d001852490faddb3018a8233c11"
format_stamp: "Formatted at 2026-08-10 07:19:44 on dist-test-slave-x5fp"
server_key: "71efd15b3cbd46e2b8eecd44110225ee"
server_key_iv: "ce1b2ef27c4afa5318f9e57dfaddc38a"
server_key_version: "encryptionkey@0"
I20260810 07:19:44.713095 20702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:44.731094 20702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:44.733739 20702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:44.734001 20702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:44.738646 20702 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:33677
I20260810 07:19:44.738689 20821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:33677 every 8 connection(s)
I20260810 07:19:44.739262 20702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:44.740067 20822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:44.741453 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 20702
I20260810 07:19:44.741556 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:44.741813 14992 external_mini_cluster.cc:1428] Setting key 5bc5fb7116976cc892c4e76e3b280fc4
I20260810 07:19:44.743242 20822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11: Bootstrap starting.
I20260810 07:19:44.744086 20822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:44.744441 20822 log.cc:824] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:44.745247 20822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11: No bootstrap required, opened a new log
I20260810 07:19:44.748335 20822 raft_consensus.cc:348] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fefb6d001852490faddb3018a8233c11" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33677 } }
I20260810 07:19:44.748574 20822 raft_consensus.cc:374] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:44.748628 20822 raft_consensus.cc:729] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fefb6d001852490faddb3018a8233c11, State: Initialized, Role: FOLLOWER
I20260810 07:19:44.748844 20822 consensus_queue.cc:260] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fefb6d001852490faddb3018a8233c11" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33677 } }
I20260810 07:19:44.748976 20822 raft_consensus.cc:388] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:44.749014 20822 raft_consensus.cc:482] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:44.749060 20822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:44.749795 20822 raft_consensus.cc:504] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fefb6d001852490faddb3018a8233c11" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33677 } }
Aug 10 07:19:44 dist-test-slave-x5fp krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:44.749904 20822 leader_election.cc:302] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fefb6d001852490faddb3018a8233c11; no voters:
I20260810 07:19:44.750181 20822 leader_election.cc:288] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:44.750213 20829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:44.750344 20829 raft_consensus.cc:686] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [term 1 LEADER]: Becoming Leader. State: Replica: fefb6d001852490faddb3018a8233c11, State: Running, Role: LEADER
I20260810 07:19:44.750471 20829 consensus_queue.cc:237] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fefb6d001852490faddb3018a8233c11" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33677 } }
I20260810 07:19:44.750546 20822 sys_catalog.cc:564] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:44.750870 20829 sys_catalog.cc:455] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fefb6d001852490faddb3018a8233c11. Latest consensus state: current_term: 1 leader_uuid: "fefb6d001852490faddb3018a8233c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fefb6d001852490faddb3018a8233c11" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33677 } } }
I20260810 07:19:44.750954 20829 sys_catalog.cc:458] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:44.751252 20830 sys_catalog.cc:455] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fefb6d001852490faddb3018a8233c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fefb6d001852490faddb3018a8233c11" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33677 } } }
I20260810 07:19:44.751343 20830 sys_catalog.cc:458] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:44.752645 20837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:44.753268 20837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:44.755741 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:44.742534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.14.164.62:33677)
0810 07:19:44.742722 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:44.742731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:44.742759 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:19:44.743218 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:44.743225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:44.743274 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:44.743453 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:44.744314 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:44.745445 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:44.746652 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:44.747146 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:44.747239 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:44.750593 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:44.750599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:44.750599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:44.750615 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:44.753341 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:44.753975 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:44.753980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:44.753980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:44.754099 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:44.754390 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:44.754391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:44.754391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:44.754755 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:44.754923 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:44.755352 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 07:19:44.755519 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260810 07:19:44.755896 20837 catalog_manager.cc:1347] Generated new cluster ID: 91f8b50756914943ae1832cd862a5ab6
I20260810 07:19:44.755945 20837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:44.798597 20837 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:44.799273 20837 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:44.804337 20837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fefb6d001852490faddb3018a8233c11: Generated new TSK 0
I20260810 07:19:44.804579 20837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:44.840392 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33677
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:44.932463 20862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:44.932680 20862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:44.932704 20862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:44.933908 20862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:44.933959 20862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:44.933985 20862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:44.935832 20862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:44.936030 20862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:44.936221 20862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:44.941538 20867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:44.941547 20870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:44.941583 20868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:44.942067 20862 server_base.cc:1034] running on GCE node
I20260810 07:19:44.942312 20862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:44.942615 20862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:44.943805 20862 hybrid_clock.cc:648] HybridClock initialized: now 1786346384943768 us; error 46 us; skew 500 ppm
Aug 10 07:19:44 dist-test-slave-x5fp krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346384, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:44.946856 20862 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:44.947355 20862 webserver.cc:460] Webserver started at http://127.14.164.1:43467/ using document root <none> and password file <none>
I20260810 07:19:44.947628 20862 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:44.947677 20862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:44.947813 20862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:44.949024 20862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "17b45f42215d47dea0199f00f3d1158b"
format_stamp: "Formatted at 2026-08-10 07:19:44 on dist-test-slave-x5fp"
server_key: "5df20cac0846d4bdf18c5b604864d8f2"
server_key_iv: "04e809cc342bedbe2cc36ca845f89947"
server_key_version: "encryptionkey@0"
I20260810 07:19:44.949424 20862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "17b45f42215d47dea0199f00f3d1158b"
format_stamp: "Formatted at 2026-08-10 07:19:44 on dist-test-slave-x5fp"
server_key: "5df20cac0846d4bdf18c5b604864d8f2"
server_key_iv: "04e809cc342bedbe2cc36ca845f89947"
server_key_version: "encryptionkey@0"
I20260810 07:19:44.950948 20862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:44.951970 20881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:44.952224 20862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:44.952342 20862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "17b45f42215d47dea0199f00f3d1158b"
format_stamp: "Formatted at 2026-08-10 07:19:44 on dist-test-slave-x5fp"
server_key: "5df20cac0846d4bdf18c5b604864d8f2"
server_key_iv: "04e809cc342bedbe2cc36ca845f89947"
server_key_version: "encryptionkey@0"
I20260810 07:19:44.952433 20862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:44.968339 20862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:44.970803 20862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:44.970978 20862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:44.971259 20862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:44.971604 20862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:44.971668 20862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:44.971719 20862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:44.971755 20862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:44.978041 20862 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:46463
I20260810 07:19:44.978114 21076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:46463 every 8 connection(s)
I20260810 07:19:44.978484 20862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:19:44 dist-test-slave-x5fp krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346384, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:44.987697 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 20862
I20260810 07:19:44.987845 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:44.988125 14992 external_mini_cluster.cc:1428] Setting key 77d82686226cfe97dba6714a624ef2d8
I20260810 07:19:44.991181 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:44.979506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:60009 (local address 127.14.164.62:33677)
0810 07:19:44.979555 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:44.979557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:44.979629 (+ 72us) server_negotiation.cc:407] Connection header received
0810 07:19:44.980959 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:44.980961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:44.980977 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:44.981063 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:44.982146 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:44.983034 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:44.984414 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:44.984691 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:44.984863 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:44.987411 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:44.987414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:44.987414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:44.987454 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:44.989486 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:44.989958 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:44.989959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:44.989960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:44.990035 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:44.990325 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:44.990326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:44.990326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:44.990493 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:44.990573 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:44.990900 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 07:19:44.991008 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:44.991505 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:44.979324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33677 (local address 127.14.164.1:60009)
0810 07:19:44.979501 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:44.979548 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:44.980784 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:44.981162 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:44.981168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:44.981676 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:44.981974 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:44.981981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:44.983176 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:44.983178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:44.984306 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:44.984309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:44.984881 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:44.984882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:44.985016 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:44.985603 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:44.985613 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:44.987249 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:44.989595 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:44.989603 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:44.989606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:44.989870 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:44.990122 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:44.990123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:44.990123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:44.990251 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:44.990590 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:44.990597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:44.990735 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 07:19:44.990847 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 07:19:44.991038 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260810 07:19:44.992123 21079 heartbeater.cc:347] Connected to a master server at 127.14.164.62:33677
I20260810 07:19:44.992256 21079 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:44.992496 21079 heartbeater.cc:511] Master 127.14.164.62:33677 requested a full tablet report, sending...
I20260810 07:19:44.993131 20765 ts_manager.cc:194] Registered new tserver with Master: 17b45f42215d47dea0199f00f3d1158b (127.14.164.1:46463)
I20260810 07:19:44.993760 20765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:60009
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:45.025130 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33677
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:45.116822 21085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:45.117066 21085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:45.117095 21085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:45.118431 21085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:45.118479 21085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:45.118508 21085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:45.120236 21085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:45.120524 21085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:45.120867 21085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:45.126619 21097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:45.126619 21091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:45.126621 21093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:45.126647 21085 server_base.cc:1034] running on GCE node
I20260810 07:19:45.127177 21085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:45.127529 21085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:45.128773 21085 hybrid_clock.cc:648] HybridClock initialized: now 1786346385128699 us; error 37 us; skew 500 ppm
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:45.131482 21085 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:45.132018 21085 webserver.cc:460] Webserver started at http://127.14.164.2:40543/ using document root <none> and password file <none>
I20260810 07:19:45.132319 21085 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:45.132369 21085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:45.132543 21085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:45.133744 21085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "5bb8576b165a4d37a4c171c836aec3f7"
format_stamp: "Formatted at 2026-08-10 07:19:45 on dist-test-slave-x5fp"
server_key: "fefd863c7031413ae3488314738d2e7a"
server_key_iv: "9aa6695253d1e8f7159e1687e0bbe878"
server_key_version: "encryptionkey@0"
I20260810 07:19:45.134123 21085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "5bb8576b165a4d37a4c171c836aec3f7"
format_stamp: "Formatted at 2026-08-10 07:19:45 on dist-test-slave-x5fp"
server_key: "fefd863c7031413ae3488314738d2e7a"
server_key_iv: "9aa6695253d1e8f7159e1687e0bbe878"
server_key_version: "encryptionkey@0"
I20260810 07:19:45.135602 21085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:45.136559 21108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:45.136853 21085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:45.136984 21085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "5bb8576b165a4d37a4c171c836aec3f7"
format_stamp: "Formatted at 2026-08-10 07:19:45 on dist-test-slave-x5fp"
server_key: "fefd863c7031413ae3488314738d2e7a"
server_key_iv: "9aa6695253d1e8f7159e1687e0bbe878"
server_key_version: "encryptionkey@0"
I20260810 07:19:45.137094 21085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:45.151574 21085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:45.154038 21085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:45.154224 21085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:45.154498 21085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:45.154891 21085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:45.154943 21085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:45.154980 21085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:45.154994 21085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:45.161538 21085 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:39797
I20260810 07:19:45.161669 21306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:39797 every 8 connection(s)
I20260810 07:19:45.161991 21085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:45.162768 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 21085
I20260810 07:19:45.162899 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:45.163151 14992 external_mini_cluster.cc:1428] Setting key d4d7ac165a1b6b10c962a93e59a70450
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
I20260810 07:19:45.173508 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:45.162893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:37773 (local address 127.14.164.62:33677)
0810 07:19:45.162969 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:45.162971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:45.163245 (+ 274us) server_negotiation.cc:407] Connection header received
0810 07:19:45.164532 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:45.164533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:45.164542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:45.164584 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:45.165537 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.166035 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.167121 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.167324 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.167413 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:45.170116 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:45.170118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:45.170119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:45.170126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:45.171746 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.172252 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.172254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.172254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.172318 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.172614 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.172615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.172615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.172803 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:45.172892 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:45.173318 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 07:19:45.173416 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:45.173629 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:45.162862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33677 (local address 127.14.164.2:37773)
0810 07:19:45.163084 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:45.163141 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:45.164416 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:45.164676 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:45.164680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:45.165132 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:45.165434 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:45.165441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.166149 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.166150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:45.167023 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:45.167026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.167424 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.167425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:45.167559 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:45.168197 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:45.168207 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:45.169986 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:45.171863 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:45.171873 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:45.171876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:45.172158 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:45.172388 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:45.172389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:45.172389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:45.172539 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:45.172887 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:45.172895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:45.173051 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 07:19:45.173228 (+ 177us) client_negotiation.cc:240] Negotiation successful
0810 07:19:45.173399 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
I20260810 07:19:45.174199 21307 heartbeater.cc:347] Connected to a master server at 127.14.164.62:33677
I20260810 07:19:45.174322 21307 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:45.174520 21307 heartbeater.cc:511] Master 127.14.164.62:33677 requested a full tablet report, sending...
I20260810 07:19:45.175027 20765 ts_manager.cc:194] Registered new tserver with Master: 5bb8576b165a4d37a4c171c836aec3f7 (127.14.164.2:39797)
I20260810 07:19:45.175478 20765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:37773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:45.197089 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33677
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:45.286082 21317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:45.286309 21317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:45.286334 21317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:45.287532 21317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:45.287578 21317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:45.287606 21317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:45.289294 21317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:45.289568 21317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:45.289826 21317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:45.299322 21328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:45.299428 21323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:45.299322 21324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:45.299731 21317 server_base.cc:1034] running on GCE node
I20260810 07:19:45.299985 21317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:45.300258 21317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:45.301450 21317 hybrid_clock.cc:648] HybridClock initialized: now 1786346385301430 us; error 33 us; skew 500 ppm
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:45.304183 21317 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:45.304708 21317 webserver.cc:460] Webserver started at http://127.14.164.3:36129/ using document root <none> and password file <none>
I20260810 07:19:45.305013 21317 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:45.305073 21317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:45.305240 21317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:45.306351 21317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "7aa343ae46764371b5b972927b513044"
format_stamp: "Formatted at 2026-08-10 07:19:45 on dist-test-slave-x5fp"
server_key: "1c3d6112a5642a5bb072d59b85f1cd47"
server_key_iv: "17cb7283cb103af79884dfff264154df"
server_key_version: "encryptionkey@0"
I20260810 07:19:45.306744 21317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "7aa343ae46764371b5b972927b513044"
format_stamp: "Formatted at 2026-08-10 07:19:45 on dist-test-slave-x5fp"
server_key: "1c3d6112a5642a5bb072d59b85f1cd47"
server_key_iv: "17cb7283cb103af79884dfff264154df"
server_key_version: "encryptionkey@0"
I20260810 07:19:45.308149 21317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:45.309092 21338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:45.309376 21317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:45.309469 21317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "7aa343ae46764371b5b972927b513044"
format_stamp: "Formatted at 2026-08-10 07:19:45 on dist-test-slave-x5fp"
server_key: "1c3d6112a5642a5bb072d59b85f1cd47"
server_key_iv: "17cb7283cb103af79884dfff264154df"
server_key_version: "encryptionkey@0"
I20260810 07:19:45.309577 21317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:45.331053 21317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:45.333629 21317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:45.333802 21317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:45.334069 21317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:45.334414 21317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:45.334472 21317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:45.334517 21317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:45.334560 21317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:45.341174 21317 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:33923
I20260810 07:19:45.341221 21536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:33923 every 8 connection(s)
I20260810 07:19:45.341570 21317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:45.349102 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 21317
I20260810 07:19:45.349217 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:45.349469 14992 external_mini_cluster.cc:1428] Setting key 36174b388f4e00719a58ffb1afdbe76d
I20260810 07:19:45.352797 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:45.342377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:56663 (local address 127.14.164.62:33677)
0810 07:19:45.342492 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:45.342495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:45.342715 (+ 220us) server_negotiation.cc:407] Connection header received
0810 07:19:45.343854 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:45.343855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:45.343871 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:45.343929 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:45.344657 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.345254 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.346383 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.346610 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.346708 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:45.349388 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:45.349390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:45.349390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:45.349397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:45.351317 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.351825 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.351826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.351827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.351883 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.352129 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.352130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.352130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.352237 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:45.352288 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:45.352591 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 07:19:45.352658 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 07:19:45.353039 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:45.342277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33677 (local address 127.14.164.3:56663)
0810 07:19:45.342546 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:45.342606 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:45.343686 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:45.344022 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:45.344034 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:45.344333 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:45.344542 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:45.344547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.345368 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.345369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:45.346255 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:45.346257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.346721 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.346723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:45.346868 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:45.347501 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:45.347511 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:45.349276 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:45.351408 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:45.351416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:45.351420 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:45.351758 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:45.351948 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:45.351949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:45.351949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:45.352069 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:45.352295 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:45.352300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:45.352468 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 07:19:45.352564 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:19:45.352705 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":109,"threads_started":1}
I20260810 07:19:45.353554 21538 heartbeater.cc:347] Connected to a master server at 127.14.164.62:33677
I20260810 07:19:45.353677 21538 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:45.353878 21538 heartbeater.cc:511] Master 127.14.164.62:33677 requested a full tablet report, sending...
I20260810 07:19:45.354306 20765 ts_manager.cc:194] Registered new tserver with Master: 7aa343ae46764371b5b972927b513044 (127.14.164.3:33923)
I20260810 07:19:45.354689 20765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:56663
I20260810 07:19:45.361577 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:45.388851 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:45.379572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.14.164.1:46463)
0810 07:19:45.379759 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:45.379770 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:45.379815 (+ 45us) server_negotiation.cc:407] Connection header received
0810 07:19:45.379845 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:45.379847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:45.379863 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:45.379949 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:45.380628 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.381471 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.382371 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.382799 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.382869 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:45.385559 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:45.385573 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:45.385574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:45.385588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:45.387464 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.387875 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.387880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.387880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.387955 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.388184 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.388185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.388186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.388356 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:45.388543 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:45.388590 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 07:19:45.388679 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:45.397972 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:45.390285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.14.164.62:33677)
0810 07:19:45.390368 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:45.390370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:45.390379 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:45.390424 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:45.390424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:45.390434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:45.390480 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:45.390937 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.391437 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.392156 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:45.392362 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:45.392438 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:45.394883 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:45.394885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:45.394885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:45.394893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:45.396552 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.396977 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.396977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.396978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.397033 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:45.397266 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:45.397267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:45.397268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:45.397443 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:45.397539 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:45.397707 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 07:19:45.397818 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 07:19:45.398250 20765 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50824
I20260810 07:19:45.398654 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 20862
I20260810 07:19:45.416523 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 21085
I20260810 07:19:45.429149 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 21317
I20260810 07:19:45.444474 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 20702
2026-08-10T07:19:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2246 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[21553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[21553](info): set up 2 sockets
Aug 10 07:19:45 dist-test-slave-x5fp krb5kdc[21553](info): commencing operation
krb5kdc: starting...
W20260810 07:19:46.751971 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:46 dist-test-slave-x5fp krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:46.884390 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:44495
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:44495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:46.970870 21575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:46.971096 21575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:46.971120 21575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:46.972325 21575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:46.972374 21575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:46.972389 21575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:46.972402 21575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:46.972469 21575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:46.974081 21575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:44495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:44495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:46.974354 21575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:46.974622 21575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:46.979874 21588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:46.979873 21583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:46.979902 21582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:46.979965 21575 server_base.cc:1034] running on GCE node
I20260810 07:19:46.980367 21575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:46.980759 21575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:46.981942 21575 hybrid_clock.cc:648] HybridClock initialized: now 1786346386981924 us; error 42 us; skew 500 ppm
Aug 10 07:19:46 dist-test-slave-x5fp krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346386, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:46.984812 21575 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:46.985213 21575 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:46.985297 21575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:46.985458 21575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:46.986616 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "14c8c86dac6a4407b46c17c77cc049ec"
format_stamp: "Formatted at 2026-08-10 07:19:46 on dist-test-slave-x5fp"
server_key: "126297bd87a64c842e79a990877f4761"
server_key_iv: "e8a0e4507a5a6a885274ed71117660cd"
server_key_version: "encryptionkey@0"
I20260810 07:19:46.987015 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "14c8c86dac6a4407b46c17c77cc049ec"
format_stamp: "Formatted at 2026-08-10 07:19:46 on dist-test-slave-x5fp"
server_key: "126297bd87a64c842e79a990877f4761"
server_key_iv: "e8a0e4507a5a6a885274ed71117660cd"
server_key_version: "encryptionkey@0"
I20260810 07:19:46.988433 21575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:46.989794 21597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:46.989993 21575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:46.990144 21575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "14c8c86dac6a4407b46c17c77cc049ec"
format_stamp: "Formatted at 2026-08-10 07:19:46 on dist-test-slave-x5fp"
server_key: "126297bd87a64c842e79a990877f4761"
server_key_iv: "e8a0e4507a5a6a885274ed71117660cd"
server_key_version: "encryptionkey@0"
I20260810 07:19:46.990236 21575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:47.011116 21575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:47.013852 21575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:47.013990 21575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:47.019342 21575 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:44495
I20260810 07:19:47.019515 21687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:44495 every 8 connection(s)
I20260810 07:19:47.019744 21575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:47.020639 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 21575
I20260810 07:19:47.020787 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:47.020994 14992 external_mini_cluster.cc:1428] Setting key 3848bd97ad8c66ae045383baad556d4b
I20260810 07:19:47.021001 21689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:47.024674 21689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec: Bootstrap starting.
I20260810 07:19:47.025507 21689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:47.025831 21689 log.cc:824] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:47.026585 21689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec: No bootstrap required, opened a new log
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:47.028851 21689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c8c86dac6a4407b46c17c77cc049ec" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44495 } }
I20260810 07:19:47.029000 21689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:47.029024 21689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c8c86dac6a4407b46c17c77cc049ec, State: Initialized, Role: FOLLOWER
I20260810 07:19:47.029165 21689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c8c86dac6a4407b46c17c77cc049ec" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44495 } }
I20260810 07:19:47.029241 21689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:47.029295 21689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:47.029343 21689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:47.030078 21689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c8c86dac6a4407b46c17c77cc049ec" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44495 } }
I20260810 07:19:47.030194 21689 leader_election.cc:302] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14c8c86dac6a4407b46c17c77cc049ec; no voters:
I20260810 07:19:47.030387 21689 leader_election.cc:288] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:47.030418 21698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:47.030530 21698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [term 1 LEADER]: Becoming Leader. State: Replica: 14c8c86dac6a4407b46c17c77cc049ec, State: Running, Role: LEADER
I20260810 07:19:47.030684 21689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:47.030700 21698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c8c86dac6a4407b46c17c77cc049ec" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44495 } }
I20260810 07:19:47.031376 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14c8c86dac6a4407b46c17c77cc049ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c8c86dac6a4407b46c17c77cc049ec" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44495 } } }
I20260810 07:19:47.031623 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:47.031544 21701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14c8c86dac6a4407b46c17c77cc049ec. Latest consensus state: current_term: 1 leader_uuid: "14c8c86dac6a4407b46c17c77cc049ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c8c86dac6a4407b46c17c77cc049ec" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 44495 } } }
I20260810 07:19:47.031908 21709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:47.032523 21709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:47.031894 21701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:47.033627 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.021644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.14.164.62:44495)
0810 07:19:47.021819 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.021828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.021866 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:19:47.022316 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.022323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.022372 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.022503 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:47.023377 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.024310 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.025411 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.025902 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.026083 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.028982 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:47.028986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:47.028986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:47.029016 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:47.031221 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.031659 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.031663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.031663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.031780 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.032087 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.032088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.032089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.032393 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:47.032568 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.033253 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.033401 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260810 07:19:47.034723 21709 catalog_manager.cc:1347] Generated new cluster ID: 5135ee649a9144bea344a2603414f07c
I20260810 07:19:47.034789 21709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:47.039058 21709 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:47.039805 21709 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:47.045063 21709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec: Generated new TSK 0
I20260810 07:19:47.045270 21709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:47.086015 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44495
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:47.176234 21733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:47.176460 21733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:47.176486 21733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:47.177742 21733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:47.177786 21733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:47.177814 21733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:47.179668 21733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:47.179962 21733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:47.180238 21733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:47.185655 21733 server_base.cc:1034] running on GCE node
W20260810 07:19:47.185637 21749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.185637 21744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.185636 21743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:47.186185 21733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:47.186479 21733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:47.187634 21733 hybrid_clock.cc:648] HybridClock initialized: now 1786346387187619 us; error 29 us; skew 500 ppm
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:47.190423 21733 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:47.190765 21733 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:47.190814 21733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:47.190948 21733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:47.192080 21733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "af12ff21ee784c0f8359bfeacddbbc89"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "6fb4551286e3100c8f679901cdfcbe84"
server_key_iv: "cf97a68e0f4067b7024d8b7710f94bd0"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.192480 21733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "af12ff21ee784c0f8359bfeacddbbc89"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "6fb4551286e3100c8f679901cdfcbe84"
server_key_iv: "cf97a68e0f4067b7024d8b7710f94bd0"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.194085 21733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:47.194944 21758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.195165 21733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:47.195261 21733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "af12ff21ee784c0f8359bfeacddbbc89"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "6fb4551286e3100c8f679901cdfcbe84"
server_key_iv: "cf97a68e0f4067b7024d8b7710f94bd0"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.195322 21733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:47.212291 21733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:47.214805 21733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:47.214939 21733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:47.215152 21733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:47.215514 21733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:47.215553 21733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.215577 21733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:47.215590 21733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.222116 21733 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:43861
I20260810 07:19:47.222195 21950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:43861 every 8 connection(s)
I20260810 07:19:47.222529 21733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:47.223105 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 21733
I20260810 07:19:47.223229 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:47.223420 14992 external_mini_cluster.cc:1428] Setting key 459e7f38acc93a26a54db32be7d694ae
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
I20260810 07:19:47.234664 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.224037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:35131 (local address 127.14.164.62:44495)
0810 07:19:47.224081 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.224083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.224360 (+ 277us) server_negotiation.cc:407] Connection header received
0810 07:19:47.224903 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.224905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.224919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.224992 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:47.226036 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.226565 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.227625 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.227851 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.227948 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.230333 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:47.230335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:47.230335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:47.230344 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:47.232393 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.233350 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.233353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.233353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.233445 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.233841 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.233842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.233842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.233998 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:47.234083 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.234423 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.234545 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:47.234937 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.224041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44495 (local address 127.14.164.1:35131)
0810 07:19:47.224218 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.224265 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.224805 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.225049 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.225053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.225349 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:47.225909 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.225915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.226674 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.226675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.227527 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.227529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.227962 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.227963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.228107 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.228706 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:47.228756 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:47.230220 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:47.232524 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:47.232534 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:47.232538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:47.233200 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:47.233561 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:47.233562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:47.233562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:47.233709 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:47.234084 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:47.234093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:47.234260 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.234381 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.234641 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260810 07:19:47.235579 21954 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44495
I20260810 07:19:47.235721 21954 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:47.236009 21954 heartbeater.cc:511] Master 127.14.164.62:44495 requested a full tablet report, sending...
I20260810 07:19:47.236850 21623 ts_manager.cc:194] Registered new tserver with Master: af12ff21ee784c0f8359bfeacddbbc89 (127.14.164.1:43861)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
I20260810 07:19:47.237902 21623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:35131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:47.257496 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44495
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:47.348376 21961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:47.348611 21961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:47.348637 21961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:47.349864 21961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:47.349916 21961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:47.349946 21961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:47.351481 21961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:47.351698 21961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:47.351903 21961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:47.357530 21971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.357523 21972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:47.357714 21961 server_base.cc:1034] running on GCE node
W20260810 07:19:47.357527 21975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:47.358116 21961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:47.358421 21961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:47.359659 21961 hybrid_clock.cc:648] HybridClock initialized: now 1786346387359620 us; error 50 us; skew 500 ppm
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:47.362663 21961 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:47.363085 21961 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:47.363160 21961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:47.363322 21961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:47.364491 21961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "62c6cb286fb84ab5bef837e262e709fa"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "a701e712538bc59f5e4f8e6697d0a876"
server_key_iv: "6ec9b64d8707b4c8d68e1e15e57ec91f"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.364928 21961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "62c6cb286fb84ab5bef837e262e709fa"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "a701e712538bc59f5e4f8e6697d0a876"
server_key_iv: "6ec9b64d8707b4c8d68e1e15e57ec91f"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.366406 21961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:47.367367 21985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.367637 21961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:47.367784 21961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "62c6cb286fb84ab5bef837e262e709fa"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "a701e712538bc59f5e4f8e6697d0a876"
server_key_iv: "6ec9b64d8707b4c8d68e1e15e57ec91f"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.367884 21961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:47.377566 21961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:47.380231 21961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:47.380446 21961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:47.380815 21961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:47.381266 21961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:47.381332 21961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.381382 21961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:47.381418 21961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.389200 21961 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:35361
I20260810 07:19:47.389240 22176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:35361 every 8 connection(s)
I20260810 07:19:47.389598 21961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:47.394896 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 21961
I20260810 07:19:47.395040 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:47.395255 14992 external_mini_cluster.cc:1428] Setting key 8d2bcd3879a1efb57465a44cbdfa825c
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:47.401659 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.390627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:33383 (local address 127.14.164.62:44495)
0810 07:19:47.390715 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.390718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.390979 (+ 261us) server_negotiation.cc:407] Connection header received
0810 07:19:47.391523 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.391524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.391537 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.391614 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:47.392819 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.393409 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.394578 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.394874 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.394984 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.398107 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:47.398110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:47.398110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:47.398120 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:47.399915 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.400422 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.400423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.400424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.400497 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.400868 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.400869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.400870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.401019 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:47.401102 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.401445 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.401547 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:19:47.401947 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.390614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44495 (local address 127.14.164.2:33383)
0810 07:19:47.390827 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.390876 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.391405 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.391721 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.391725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.392071 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:47.392638 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.392644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.393518 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.393519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.394463 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.394466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.395008 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.395009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.395182 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.395996 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:47.396006 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:47.397977 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:47.400031 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:47.400040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:47.400043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:47.400328 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:47.400593 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:47.400594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:47.400594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:47.400788 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:47.401105 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:47.401115 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:47.401290 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.401421 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.401615 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260810 07:19:47.402586 22177 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44495
I20260810 07:19:47.402717 22177 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:47.402930 22177 heartbeater.cc:511] Master 127.14.164.62:44495 requested a full tablet report, sending...
I20260810 07:19:47.403438 21623 ts_manager.cc:194] Registered new tserver with Master: 62c6cb286fb84ab5bef837e262e709fa (127.14.164.2:35361)
I20260810 07:19:47.403913 21623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:33383
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:47.429551 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44495
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:47.519837 22184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:47.520095 22184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:47.520123 22184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:47.521690 22184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:47.521768 22184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:47.521811 22184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:47.523878 22184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:44495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:47.524137 22184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:47.524338 22184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:47.530812 22184 server_base.cc:1034] running on GCE node
W20260810 07:19:47.530761 22191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.530776 22194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.530761 22190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:47.531296 22184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:47.531644 22184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:47.532841 22184 hybrid_clock.cc:648] HybridClock initialized: now 1786346387532808 us; error 49 us; skew 500 ppm
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:47.539094 22184 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:47.539543 22184 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:47.539601 22184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:47.539746 22184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:47.540997 22184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "f7b1561bc5c94042a3cc7482c848ae1c"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "4028ee9603aa54635cb0fc8e2aca7191"
server_key_iv: "5bf58ae14553e6b6588b47ef9616e1f5"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.541389 22184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "f7b1561bc5c94042a3cc7482c848ae1c"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "4028ee9603aa54635cb0fc8e2aca7191"
server_key_iv: "5bf58ae14553e6b6588b47ef9616e1f5"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.542812 22184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:47.543639 22206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.543862 22184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:47.544000 22184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "f7b1561bc5c94042a3cc7482c848ae1c"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "4028ee9603aa54635cb0fc8e2aca7191"
server_key_iv: "5bf58ae14553e6b6588b47ef9616e1f5"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.544077 22184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:47.557559 22184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:47.559926 22184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:47.560109 22184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:47.560413 22184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:47.560853 22184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:47.560895 22184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.560922 22184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:47.560935 22184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.567517 22184 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:44665
I20260810 07:19:47.567618 22406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:44665 every 8 connection(s)
I20260810 07:19:47.567898 22184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:19:47 dist-test-slave-x5fp krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:47.576989 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 22184
I20260810 07:19:47.577216 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:47.577457 14992 external_mini_cluster.cc:1428] Setting key 6a02c4bc29807e49769ad6a400e05bbb
I20260810 07:19:47.579747 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.568690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:34633 (local address 127.14.164.62:44495)
0810 07:19:47.568775 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.568777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.569108 (+ 331us) server_negotiation.cc:407] Connection header received
0810 07:19:47.569622 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.569624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.569637 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.569718 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:47.570752 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.571506 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.572697 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.573035 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.573112 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.575760 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:47.575792 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:47.575792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:47.575801 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:47.577928 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.578521 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.578522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.578523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.578596 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.578891 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.578893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.578893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.579167 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:47.579251 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.579586 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.579667 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:47.579917 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.568795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:44495 (local address 127.14.164.3:34633)
0810 07:19:47.568967 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.569007 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.569500 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.569775 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.569778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.570082 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:47.570621 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.570626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.571625 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.571626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.572586 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.572589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.573158 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.573159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.573320 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.574013 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:47.574028 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:47.575649 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:47.578065 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:47.578075 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:47.578078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:47.578397 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:47.578668 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:47.578674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:47.578674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:47.578800 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:47.579270 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:47.579277 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:47.579419 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.579519 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.579656 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260810 07:19:47.580818 22407 heartbeater.cc:347] Connected to a master server at 127.14.164.62:44495
I20260810 07:19:47.580943 22407 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:47.581144 22407 heartbeater.cc:511] Master 127.14.164.62:44495 requested a full tablet report, sending...
I20260810 07:19:47.581593 21623 ts_manager.cc:194] Registered new tserver with Master: f7b1561bc5c94042a3cc7482c848ae1c (127.14.164.3:44665)
I20260810 07:19:47.581970 21623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:34633
I20260810 07:19:47.589627 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:47.598047 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.591196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.14.164.62:44495)
0810 07:19:47.591316 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.591319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.591393 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:19:47.591492 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.591494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.591504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.591564 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:47.592196 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.592769 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.593557 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.593818 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.593905 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.595234 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:47.595235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:47.595235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:47.595243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:47.596709 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.597107 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.597109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.597109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.597169 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:47.597385 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:47.597386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:47.597386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:47.597554 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:47.597670 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.597855 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.597939 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:19:47.599874 21623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:47.601668 21623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:47.609539 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.604398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:43861 (local address 127.0.0.1:34132)
0810 07:19:47.604589 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.604638 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.604738 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.605585 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.605586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.605589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:47.605637 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.605638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.606644 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.606645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.608157 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.608160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.609177 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.609178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.609272 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.609290 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.609370 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.609431 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":7,"thread_start_us":121,"threads_started":1}
I20260810 07:19:47.609840 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.604705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.14.164.1:43861)
0810 07:19:47.604947 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.604953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.604967 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:47.605016 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.605018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.605037 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.605703 (+ 666us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:47.605776 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.606531 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.608264 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.609084 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.609166 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.609230 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.609693 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.609751 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":115,"threads_started":1}
I20260810 07:19:47.610250 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.604402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:35361 (local address 127.0.0.1:52604)
0810 07:19:47.605146 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.605157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.605210 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.605457 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.605458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.605460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:47.605511 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.605512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.606887 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.606888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.608156 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.608160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.609962 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.609964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.610046 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.610052 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.610121 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.610172 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":5,"thread_start_us":121,"threads_started":1}
I20260810 07:19:47.610293 22432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.604626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.14.164.2:35361)
0810 07:19:47.604849 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.604854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.605301 (+ 447us) server_negotiation.cc:407] Connection header received
0810 07:19:47.605352 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.605355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.605373 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.605460 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:47.605629 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.606760 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.608270 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.609084 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.609166 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.609295 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.610155 (+ 860us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.610203 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":81,"threads_started":1}
I20260810 07:19:47.610780 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.604398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:44665 (local address 127.0.0.1:58670)
0810 07:19:47.604970 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.604983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.605057 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.605405 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.605409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.605411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:47.605494 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.605496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.608288 (+ 2792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.608289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.609116 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.609118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.610479 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.610480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.610554 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.610564 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.610648 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.610691 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260810 07:19:47.610797 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.604695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58670 (local address 127.14.164.3:44665)
0810 07:19:47.604956 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.604959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.605056 (+ 97us) server_negotiation.cc:407] Connection header received
0810 07:19:47.605179 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.605181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.605198 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.605278 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:47.605586 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.606769 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.609252 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.610349 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.610427 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.610490 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.610668 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.610713 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":174,"threads_started":1}
I20260810 07:19:47.610710 21835 tablet_service.cc:1467] Processing CreateTablet for tablet 227c6bc6d6cc455690b16798d0713025 (DEFAULT_TABLE table=test-table [id=674dc204f569456792eeb79c7a82c8b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:47.611138 21835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 227c6bc6d6cc455690b16798d0713025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:47.611219 22074 tablet_service.cc:1467] Processing CreateTablet for tablet 227c6bc6d6cc455690b16798d0713025 (DEFAULT_TABLE table=test-table [id=674dc204f569456792eeb79c7a82c8b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:47.611548 22074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 227c6bc6d6cc455690b16798d0713025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:47.611749 22293 tablet_service.cc:1467] Processing CreateTablet for tablet 227c6bc6d6cc455690b16798d0713025 (DEFAULT_TABLE table=test-table [id=674dc204f569456792eeb79c7a82c8b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:47.612114 22293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 227c6bc6d6cc455690b16798d0713025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:47.615237 22437 tablet_bootstrap.cc:492] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Bootstrap starting.
I20260810 07:19:47.615561 22439 tablet_bootstrap.cc:492] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89: Bootstrap starting.
I20260810 07:19:47.615984 22437 tablet_bootstrap.cc:654] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:47.616221 22439 tablet_bootstrap.cc:654] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:47.616648 22437 log.cc:824] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:47.616875 22439 log.cc:824] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:47.616910 22441 tablet_bootstrap.cc:492] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Bootstrap starting.
I20260810 07:19:47.617574 22441 tablet_bootstrap.cc:654] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:47.617636 22437 tablet_bootstrap.cc:492] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: No bootstrap required, opened a new log
I20260810 07:19:47.617766 22437 ts_tablet_manager.cc:1397] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:47.617810 22439 tablet_bootstrap.cc:492] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89: No bootstrap required, opened a new log
I20260810 07:19:47.617877 22439 ts_tablet_manager.cc:1397] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:47.618156 22441 log.cc:824] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:47.618997 22441 tablet_bootstrap.cc:492] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: No bootstrap required, opened a new log
I20260810 07:19:47.619069 22441 ts_tablet_manager.cc:1397] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:47.620154 22437 raft_consensus.cc:348] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.620362 22437 raft_consensus.cc:374] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:47.620404 22437 raft_consensus.cc:729] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b1561bc5c94042a3cc7482c848ae1c, State: Initialized, Role: FOLLOWER
I20260810 07:19:47.620584 22437 consensus_queue.cc:260] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.620584 22439 raft_consensus.cc:348] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.620590 22441 raft_consensus.cc:348] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.620813 22441 raft_consensus.cc:374] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:47.620836 22439 raft_consensus.cc:374] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:47.620862 22441 raft_consensus.cc:729] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c6cb286fb84ab5bef837e262e709fa, State: Initialized, Role: FOLLOWER
I20260810 07:19:47.620889 22439 raft_consensus.cc:729] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af12ff21ee784c0f8359bfeacddbbc89, State: Initialized, Role: FOLLOWER
I20260810 07:19:47.620990 22437 ts_tablet_manager.cc:1428] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:47.620978 22441 consensus_queue.cc:260] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.621131 22407 heartbeater.cc:503] Master 127.14.164.62:44495 was elected leader, sending a full tablet report...
I20260810 07:19:47.621122 22439 consensus_queue.cc:260] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.621263 22177 heartbeater.cc:503] Master 127.14.164.62:44495 was elected leader, sending a full tablet report...
I20260810 07:19:47.621451 21954 heartbeater.cc:503] Master 127.14.164.62:44495 was elected leader, sending a full tablet report...
I20260810 07:19:47.621250 22441 ts_tablet_manager.cc:1428] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:47.621717 22439 ts_tablet_manager.cc:1428] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
W20260810 07:19:47.640261 22179 tablet.cc:2369] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:47.672501 22448 raft_consensus.cc:482] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:47.672645 22448 raft_consensus.cc:504] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.673077 22448 leader_election.cc:288] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b1561bc5c94042a3cc7482c848ae1c (127.14.164.3:44665), 62c6cb286fb84ab5bef837e262e709fa (127.14.164.2:35361)
I20260810 07:19:47.676365 22432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.673448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:36559 (local address 127.14.164.2:35361)
0810 07:19:47.673506 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.673508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.673519 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:47.673550 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.673551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.673561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.673645 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:47.673868 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.674458 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.675604 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.675994 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.676053 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.676060 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.676273 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.676305 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:19:47.676396 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.673312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:35361 (local address 127.14.164.1:36559)
0810 07:19:47.673380 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.673396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.673471 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.673649 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.673650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.673651 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:47.673790 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.673793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.674544 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.674545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.675504 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.675507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.676093 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.676094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.676186 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.676190 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.676259 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.676315 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":26}
I20260810 07:19:47.676787 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.673407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:59417 (local address 127.14.164.3:44665)
0810 07:19:47.673490 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.673492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.673792 (+ 300us) server_negotiation.cc:407] Connection header received
0810 07:19:47.673825 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.673826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.673838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.673896 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:47.674074 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.674650 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.675646 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.676410 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.676484 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.676493 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.676653 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.676702 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:19:47.676795 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.673312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:44665 (local address 127.14.164.1:59417)
0810 07:19:47.673678 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.673691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:47.673752 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:47.673925 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:47.673926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:47.673928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:47.673999 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.674000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.674758 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.674759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.675519 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:47.675521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.676530 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.676531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:47.676593 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.676596 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:47.676643 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:19:47.676677 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":169,"thread_start_us":138,"threads_started":1}
I20260810 07:19:47.676829 22109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227c6bc6d6cc455690b16798d0713025" candidate_uuid: "af12ff21ee784c0f8359bfeacddbbc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c6cb286fb84ab5bef837e262e709fa" is_pre_election: true
I20260810 07:19:47.677016 22109 raft_consensus.cc:2454] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af12ff21ee784c0f8359bfeacddbbc89 in term 0.
I20260810 07:19:47.677258 22335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227c6bc6d6cc455690b16798d0713025" candidate_uuid: "af12ff21ee784c0f8359bfeacddbbc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" is_pre_election: true
I20260810 07:19:47.677284 21762 leader_election.cc:302] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c6cb286fb84ab5bef837e262e709fa, af12ff21ee784c0f8359bfeacddbbc89; no voters:
I20260810 07:19:47.677424 22335 raft_consensus.cc:2454] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af12ff21ee784c0f8359bfeacddbbc89 in term 0.
I20260810 07:19:47.677461 22448 raft_consensus.cc:2781] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:47.677560 22448 raft_consensus.cc:482] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:47.677597 22448 raft_consensus.cc:3037] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:47.678357 22448 raft_consensus.cc:504] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.678516 22448 leader_election.cc:288] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [CANDIDATE]: Term 1 election: Requested vote from peers f7b1561bc5c94042a3cc7482c848ae1c (127.14.164.3:44665), 62c6cb286fb84ab5bef837e262e709fa (127.14.164.2:35361)
I20260810 07:19:47.678710 22109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227c6bc6d6cc455690b16798d0713025" candidate_uuid: "af12ff21ee784c0f8359bfeacddbbc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c6cb286fb84ab5bef837e262e709fa"
I20260810 07:19:47.678805 22109 raft_consensus.cc:3037] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:47.678828 22335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227c6bc6d6cc455690b16798d0713025" candidate_uuid: "af12ff21ee784c0f8359bfeacddbbc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b1561bc5c94042a3cc7482c848ae1c"
I20260810 07:19:47.678905 22335 raft_consensus.cc:3037] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:47.679823 22335 raft_consensus.cc:2454] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af12ff21ee784c0f8359bfeacddbbc89 in term 1.
I20260810 07:19:47.679817 22109 raft_consensus.cc:2454] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af12ff21ee784c0f8359bfeacddbbc89 in term 1.
I20260810 07:19:47.680049 21762 leader_election.cc:302] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c6cb286fb84ab5bef837e262e709fa, af12ff21ee784c0f8359bfeacddbbc89; no voters:
I20260810 07:19:47.680179 22448 raft_consensus.cc:2781] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:47.680368 22448 raft_consensus.cc:686] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [term 1 LEADER]: Becoming Leader. State: Replica: af12ff21ee784c0f8359bfeacddbbc89, State: Running, Role: LEADER
I20260810 07:19:47.680475 22448 consensus_queue.cc:237] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } }
I20260810 07:19:47.681435 21623 catalog_manager.cc:5495] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 reported cstate change: term changed from 0 to 1, leader changed from <none> to af12ff21ee784c0f8359bfeacddbbc89 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "af12ff21ee784c0f8359bfeacddbbc89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af12ff21ee784c0f8359bfeacddbbc89" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 43861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:47.706576 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.702336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.14.164.1:43861)
0810 07:19:47.702425 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.702427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.702501 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:19:47.702613 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.702614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.702638 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.702732 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:47.702982 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.703730 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.705433 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:47.705772 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:47.705868 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:47.706130 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:47.706314 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:47.706400 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.706457 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.706480 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:19:47.709004 22335 raft_consensus.cc:1261] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 1 FOLLOWER]: Refusing update from remote peer af12ff21ee784c0f8359bfeacddbbc89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:47.709093 22109 raft_consensus.cc:1261] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 1 FOLLOWER]: Refusing update from remote peer af12ff21ee784c0f8359bfeacddbbc89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:47.709424 22450 consensus_queue.cc:1035] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b1561bc5c94042a3cc7482c848ae1c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 44665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:47.709542 22448 consensus_queue.cc:1035] T 227c6bc6d6cc455690b16798d0713025 P af12ff21ee784c0f8359bfeacddbbc89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c6cb286fb84ab5bef837e262e709fa" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:47.717070 22456 mvcc.cc:205] Tried to move back new op lower bound from 7316874804052631552 to 7316874803939913728. Current Snapshot: MvccSnapshot[applied={T|T < 7316874804052631552 or (T in {7316874804052631552})}]
I20260810 07:19:47.719883 21621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:47.720008 21621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:47.720912 21621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14c8c86dac6a4407b46c17c77cc049ec: Sending DeleteTablet for 3 replicas of tablet 227c6bc6d6cc455690b16798d0713025
I20260810 07:19:47.721321 22074 tablet_service.cc:1514] Processing DeleteTablet for tablet 227c6bc6d6cc455690b16798d0713025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:47 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:52604
I20260810 07:19:47.721576 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 21733
I20260810 07:19:47.721596 22466 tablet_replica.cc:330] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: stopping tablet replica
I20260810 07:19:47.721606 22293 tablet_service.cc:1514] Processing DeleteTablet for tablet 227c6bc6d6cc455690b16798d0713025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:47 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:58670
I20260810 07:19:47.721711 22466 raft_consensus.cc:2229] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:47.721833 22466 raft_consensus.cc:2258] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:47.729239 22467 tablet_replica.cc:330] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: stopping tablet replica
I20260810 07:19:47.730106 22467 raft_consensus.cc:2229] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:47.730279 22467 raft_consensus.cc:2258] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:47.732968 22467 ts_tablet_manager.cc:1905] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:47.735018 22466 ts_tablet_manager.cc:1905] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:47.735037 22467 ts_tablet_manager.cc:1918] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:47.735118 22467 log.cc:1196] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/227c6bc6d6cc455690b16798d0713025
I20260810 07:19:47.735406 22467 ts_tablet_manager.cc:1939] T 227c6bc6d6cc455690b16798d0713025 P f7b1561bc5c94042a3cc7482c848ae1c: Deleting consensus metadata
I20260810 07:19:47.735975 21600 catalog_manager.cc:4841] TS f7b1561bc5c94042a3cc7482c848ae1c (127.14.164.3:44665): tablet 227c6bc6d6cc455690b16798d0713025 (table test-table [id=674dc204f569456792eeb79c7a82c8b9]) successfully deleted
I20260810 07:19:47.736634 22466 ts_tablet_manager.cc:1918] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:47.736759 22466 log.cc:1196] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/227c6bc6d6cc455690b16798d0713025
I20260810 07:19:47.737040 22466 ts_tablet_manager.cc:1939] T 227c6bc6d6cc455690b16798d0713025 P 62c6cb286fb84ab5bef837e262e709fa: Deleting consensus metadata
I20260810 07:19:47.737496 21603 catalog_manager.cc:4841] TS 62c6cb286fb84ab5bef837e262e709fa (127.14.164.2:35361): tablet 227c6bc6d6cc455690b16798d0713025 (table test-table [id=674dc204f569456792eeb79c7a82c8b9]) successfully deleted
W20260810 07:19:47.739671 21599 connection.cc:535] client connection to 127.14.164.1:43861 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:19:47.739799 21599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:19:47.739917 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 21961
I20260810 07:19:47.757843 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.757665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:43861 (local address 127.0.0.1:34164)
0810 07:19:47.757731 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:47.757791 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:43861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:19:47.758049 21599 catalog_manager.cc:4584] TS af12ff21ee784c0f8359bfeacddbbc89 (127.14.164.1:43861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 227c6bc6d6cc455690b16798d0713025: Network error: Client connection negotiation failed: client connection to 127.14.164.1:43861: connect: Connection refused (error 111)
I20260810 07:19:47.763250 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 22184
I20260810 07:19:47.772931 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 21575
2026-08-10T07:19:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2323 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T07:19:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:47Z Disabled control of system clock
I20260810 07:19:47.810962 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:46069
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:46069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:19:47.896445 22476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:47.896667 22476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:47.896688 22476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:47.897913 22476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:47.897962 22476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:47.897975 22476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:47.897989 22476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:47.898000 22476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:47.899334 22476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:46069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:46069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:47.899552 22476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:47.899820 22476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:47.905141 22487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.905145 22489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:47.905321 22486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:47.905527 22476 server_base.cc:1034] running on GCE node
I20260810 07:19:47.905851 22476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:47.906215 22476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:47.907406 22476 hybrid_clock.cc:648] HybridClock initialized: now 1786346387907389 us; error 30 us; skew 500 ppm
I20260810 07:19:47.909015 22476 webserver.cc:460] Webserver started at http://127.14.164.62:41501/ using document root <none> and password file <none>
I20260810 07:19:47.909276 22476 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:47.909348 22476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:47.909515 22476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:47.910665 22476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "2d7c0241583f4a489fa4b84a6b8b18e1"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "227ecf42a1d7f3469e541e1685b86ce1"
server_key_iv: "b3b4c4d6c3ca621ec6581b352ee2a388"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.911062 22476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "2d7c0241583f4a489fa4b84a6b8b18e1"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "227ecf42a1d7f3469e541e1685b86ce1"
server_key_iv: "b3b4c4d6c3ca621ec6581b352ee2a388"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.912528 22476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:47.913471 22497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:47.913656 22476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:47.913770 22476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "2d7c0241583f4a489fa4b84a6b8b18e1"
format_stamp: "Formatted at 2026-08-10 07:19:47 on dist-test-slave-x5fp"
server_key: "227ecf42a1d7f3469e541e1685b86ce1"
server_key_iv: "b3b4c4d6c3ca621ec6581b352ee2a388"
server_key_version: "encryptionkey@0"
I20260810 07:19:47.913883 22476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:47.917506 22476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:47.917800 22476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:47.917953 22476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:47.922195 22476 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:46069
I20260810 07:19:47.922420 22579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:46069 every 8 connection(s)
I20260810 07:19:47.922627 22476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:47.923357 22582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:47.926155 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 22476
I20260810 07:19:47.926281 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:47.926484 14992 external_mini_cluster.cc:1428] Setting key 0854e5688bfdd96cb47e343caf9246cb
I20260810 07:19:47.927011 22582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1: Bootstrap starting.
I20260810 07:19:47.927785 22582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:47.928226 22582 log.cc:824] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:47.929302 22582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1: No bootstrap required, opened a new log
I20260810 07:19:47.929666 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:47.927186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.14.164.62:46069)
0810 07:19:47.927396 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:47.927403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:47.927433 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:19:47.927924 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:47.927932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:47.927974 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:47.928097 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:47.929139 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:47.929142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:47.929143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:47.929147 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:47.929195 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:47.929205 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:47.929290 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:47.929439 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 07:19:47.929512 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260810 07:19:47.931306 22582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46069 } }
I20260810 07:19:47.931468 22582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:47.931497 22582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d7c0241583f4a489fa4b84a6b8b18e1, State: Initialized, Role: FOLLOWER
I20260810 07:19:47.931658 22582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46069 } }
I20260810 07:19:47.932075 22582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:47.932152 22582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:47.932235 22582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:47.933132 22582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46069 } }
I20260810 07:19:47.933292 22582 leader_election.cc:302] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d7c0241583f4a489fa4b84a6b8b18e1; no voters:
I20260810 07:19:47.933501 22582 leader_election.cc:288] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:47.933547 22590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:47.933841 22582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:47.933856 22590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [term 1 LEADER]: Becoming Leader. State: Replica: 2d7c0241583f4a489fa4b84a6b8b18e1, State: Running, Role: LEADER
I20260810 07:19:47.934094 22590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46069 } }
I20260810 07:19:47.934540 22591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46069 } } }
I20260810 07:19:47.934639 22591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:47.934877 22603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:47.935030 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d7c0241583f4a489fa4b84a6b8b18e1. Latest consensus state: current_term: 1 leader_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7c0241583f4a489fa4b84a6b8b18e1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46069 } } }
I20260810 07:19:47.935159 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:47.935470 22603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:47.937904 22603 catalog_manager.cc:1347] Generated new cluster ID: 1ea961d4372b43d882ba94012b0845f9
I20260810 07:19:47.937973 22603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:47.947301 22603 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:47.947518 22603 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:47.953080 22603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1: Generated new TSK 0
I20260810 07:19:47.953338 22603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:47.959841 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:46069
--builtin_ntp_servers=127.14.164.20:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:19:48.072367 22626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.072649 22626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.072691 22626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.073927 22626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.073994 22626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.074040 22626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:48.075431 22626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:46069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:48.075760 22626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.076092 22626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.082461 22635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.082687 22626 server_base.cc:1034] running on GCE node
W20260810 07:19:48.082455 22634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.082541 22637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.083037 22626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.083360 22626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.084553 22626 hybrid_clock.cc:648] HybridClock initialized: now 1786346388084535 us; error 30 us; skew 500 ppm
I20260810 07:19:48.086172 22626 webserver.cc:460] Webserver started at http://127.14.164.1:43725/ using document root <none> and password file <none>
I20260810 07:19:48.086483 22626 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.086555 22626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.086695 22626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.088192 22626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "f76c89b7b7dc4265b0d7867ecef7203e"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "501ec7b44c8f2191755630556dfde140"
server_key_iv: "0cb7c45498fcba7238eea572f638d2c6"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.088613 22626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "f76c89b7b7dc4265b0d7867ecef7203e"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "501ec7b44c8f2191755630556dfde140"
server_key_iv: "0cb7c45498fcba7238eea572f638d2c6"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.090355 22626 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 07:19:48.091284 22646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.091470 22626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:48.091583 22626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "f76c89b7b7dc4265b0d7867ecef7203e"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "501ec7b44c8f2191755630556dfde140"
server_key_iv: "0cb7c45498fcba7238eea572f638d2c6"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.091667 22626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.095157 22626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.095458 22626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.095607 22626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.095821 22626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:48.096153 22626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:48.096211 22626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.096252 22626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:48.096282 22626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.102258 22626 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:32945
I20260810 07:19:48.102366 22837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:32945 every 8 connection(s)
I20260810 07:19:48.102720 22626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:48.105963 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.103485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:32843 (local address 127.14.164.62:46069)
0810 07:19:48.103538 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.103540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.103783 (+ 243us) server_negotiation.cc:407] Connection header received
0810 07:19:48.104325 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.104326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.104333 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.104395 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.105693 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.105695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.105695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.105700 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.105717 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.105722 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.105776 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.105872 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.105888 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:19:48.106190 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.103468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:46069 (local address 127.14.164.1:32843)
0810 07:19:48.103652 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.103687 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.104212 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.104480 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.104484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.104492 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.105526 (+ 1034us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.105526 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.105556 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.105586 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.105774 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.105780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.105783 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.105882 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.105956 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260810 07:19:48.106756 22839 heartbeater.cc:347] Connected to a master server at 127.14.164.62:46069
I20260810 07:19:48.106875 22839 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:48.107040 22839 heartbeater.cc:511] Master 127.14.164.62:46069 requested a full tablet report, sending...
I20260810 07:19:48.107649 22528 ts_manager.cc:194] Registered new tserver with Master: f76c89b7b7dc4265b0d7867ecef7203e (127.14.164.1:32945)
I20260810 07:19:48.107844 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 22626
I20260810 07:19:48.107957 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:48.108156 14992 external_mini_cluster.cc:1428] Setting key 7a34ed9e66a50bbb5f7c1a7f47d7cb6a
I20260810 07:19:48.109114 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:46069
--builtin_ntp_servers=127.14.164.20:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:19:48.193784 22842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.193994 22842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.194017 22842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.195248 22842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.195292 22842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.195319 22842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:48.196666 22842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:46069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:48.196979 22842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.197285 22842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:48.202495 22842 server_base.cc:1034] running on GCE node
W20260810 07:19:48.202477 22851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.202476 22853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.202457 22850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.202874 22842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.203145 22842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.204296 22842 hybrid_clock.cc:648] HybridClock initialized: now 1786346388204286 us; error 49 us; skew 500 ppm
I20260810 07:19:48.205694 22842 webserver.cc:460] Webserver started at http://127.14.164.2:34351/ using document root <none> and password file <none>
I20260810 07:19:48.205950 22842 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.205996 22842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.206146 22842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.207255 22842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "39488c35c84846919641c5fc421f2480"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "25f63a411628b7dee9d327e3a6d5c59a"
server_key_iv: "c7160e1ac2c0e381b92e58e9bfd30ea7"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.207628 22842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "39488c35c84846919641c5fc421f2480"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "25f63a411628b7dee9d327e3a6d5c59a"
server_key_iv: "c7160e1ac2c0e381b92e58e9bfd30ea7"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.209060 22842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:48.209872 22862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.210047 22842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:48.210142 22842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "39488c35c84846919641c5fc421f2480"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "25f63a411628b7dee9d327e3a6d5c59a"
server_key_iv: "c7160e1ac2c0e381b92e58e9bfd30ea7"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.210199 22842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.213536 22842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.213753 22842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.213845 22842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.214010 22842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:48.214247 22842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:48.214277 22842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.214298 22842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:48.214310 22842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.220137 22842 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:46847
I20260810 07:19:48.220217 23049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:46847 every 8 connection(s)
I20260810 07:19:48.220551 22842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:48.223611 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.221368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:50755 (local address 127.14.164.62:46069)
0810 07:19:48.221460 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.221462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.221699 (+ 237us) server_negotiation.cc:407] Connection header received
0810 07:19:48.222182 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.222183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.222188 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.222254 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.223400 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.223402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.223402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.223412 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.223428 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.223432 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.223480 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.223541 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.223554 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:19:48.223811 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.221338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:46069 (local address 127.14.164.2:50755)
0810 07:19:48.221566 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.221608 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.222094 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.222330 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.222333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.222342 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.223238 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.223239 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.223267 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.223312 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.223478 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.223480 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.223483 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.223533 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.223580 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":127,"threads_started":1}
I20260810 07:19:48.224331 23050 heartbeater.cc:347] Connected to a master server at 127.14.164.62:46069
I20260810 07:19:48.224432 23050 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:48.224583 23050 heartbeater.cc:511] Master 127.14.164.62:46069 requested a full tablet report, sending...
I20260810 07:19:48.224943 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 22842
I20260810 07:19:48.225054 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:48.225099 22528 ts_manager.cc:194] Registered new tserver with Master: 39488c35c84846919641c5fc421f2480 (127.14.164.2:46847)
I20260810 07:19:48.225255 14992 external_mini_cluster.cc:1428] Setting key 0fdc106b3c029df4c3f90dc98cffefb0
I20260810 07:19:48.226253 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:46069
--builtin_ntp_servers=127.14.164.20:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:19:48.311836 23055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.312038 23055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.312062 23055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.313371 23055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.313422 23055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.313449 23055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:48.314914 23055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:46069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:48.315183 23055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.315523 23055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.320740 23062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.320755 23067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.320979 23063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.320963 23055 server_base.cc:1034] running on GCE node
I20260810 07:19:48.321262 23055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.321482 23055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.322695 23055 hybrid_clock.cc:648] HybridClock initialized: now 1786346388322685 us; error 47 us; skew 500 ppm
I20260810 07:19:48.324369 23055 webserver.cc:460] Webserver started at http://127.14.164.3:43639/ using document root <none> and password file <none>
I20260810 07:19:48.324661 23055 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.324771 23055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.324929 23055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.326050 23055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "61c0f85e948b4878a9280644707fba75"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "f9fbbef804fab87b2f16a6ebb3d45af8"
server_key_iv: "70e2bcda0bdc9c1f51b2074f58bf2198"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.326436 23055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "61c0f85e948b4878a9280644707fba75"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "f9fbbef804fab87b2f16a6ebb3d45af8"
server_key_iv: "70e2bcda0bdc9c1f51b2074f58bf2198"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.327852 23055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:48.328688 23077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.328941 23055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:48.329056 23055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "61c0f85e948b4878a9280644707fba75"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "f9fbbef804fab87b2f16a6ebb3d45af8"
server_key_iv: "70e2bcda0bdc9c1f51b2074f58bf2198"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.329146 23055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.332659 23055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.333009 23055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.333303 23055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.333544 23055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:48.333917 23055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:48.333976 23055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.334019 23055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:48.334087 23055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.340225 23055 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:41961
I20260810 07:19:48.340274 23249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:41961 every 8 connection(s)
I20260810 07:19:48.340622 23055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:48.342145 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 23055
I20260810 07:19:48.342281 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:48.342478 14992 external_mini_cluster.cc:1428] Setting key d3d194d22ed09251053c8cc199fe70d2
I20260810 07:19:48.344156 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.341457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:52473 (local address 127.14.164.62:46069)
0810 07:19:48.341512 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.341514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.341755 (+ 241us) server_negotiation.cc:407] Connection header received
0810 07:19:48.342331 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.342333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.342340 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.342391 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.343890 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.343891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.343892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.343897 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.343924 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.343928 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.343988 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.344071 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.344086 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:19:48.344388 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.341434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:46069 (local address 127.14.164.3:52473)
0810 07:19:48.341625 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.341661 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.342222 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.342473 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.342478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.342488 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.343725 (+ 1237us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.343726 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.343757 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.343786 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.344000 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.344007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.344009 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.344084 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.344149 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260810 07:19:48.344941 23250 heartbeater.cc:347] Connected to a master server at 127.14.164.62:46069
I20260810 07:19:48.345031 23250 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:48.345148 23250 heartbeater.cc:511] Master 127.14.164.62:46069 requested a full tablet report, sending...
I20260810 07:19:48.345494 22528 ts_manager.cc:194] Registered new tserver with Master: 61c0f85e948b4878a9280644707fba75 (127.14.164.3:41961)
I20260810 07:19:48.354703 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:48.357730 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.356392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52556 (local address 127.14.164.62:46069)
0810 07:19:48.356471 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.356473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.356587 (+ 114us) server_negotiation.cc:407] Connection header received
0810 07:19:48.356672 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.356673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.356679 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.356782 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.357469 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.357471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.357471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.357476 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.357492 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.357496 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.357572 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.357631 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.357648 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:48.359659 22528 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:48.361431 22528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:48.365520 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.363750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:41961 (local address 127.0.0.1:47806)
0810 07:19:48.364077 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.364089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.364151 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.364489 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.364497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.364504 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.365043 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.365043 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.365058 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.365079 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.365340 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.365347 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.365350 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.365428 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.365442 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":60,"threads_started":1}
I20260810 07:19:48.365731 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.363873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.14.164.3:41961)
0810 07:19:48.364296 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.364300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.364311 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:48.364338 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.364341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.364350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.364748 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.365186 (+ 438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.365193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.365194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.365199 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.365244 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.365265 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.365332 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.365628 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.365645 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":103,"threads_started":1}
I20260810 07:19:48.365759 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.363852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.14.164.2:46847)
0810 07:19:48.364007 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.364013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.364157 (+ 144us) server_negotiation.cc:407] Connection header received
0810 07:19:48.364263 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.364267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.364278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.364349 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.365439 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.365450 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.365450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.365455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.365490 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.365521 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.365582 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.365664 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.365679 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260810 07:19:48.366119 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.363989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32945 (local address 127.0.0.1:51632)
0810 07:19:48.364112 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.364122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.364162 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.364769 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.364770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.364772 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.365431 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.365431 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.365439 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.365451 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.365975 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.365976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.365976 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.366018 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.366032 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260810 07:19:48.366202 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.363748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:46847 (local address 127.0.0.1:42898)
0810 07:19:48.364039 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.364086 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.364178 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.364685 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.364686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.364689 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.365349 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.365349 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.365357 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.365367 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.365574 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.365574 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.365575 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.366084 (+ 509us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.366099 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":136,"threads_started":1}
I20260810 07:19:48.366400 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.364208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.14.164.1:32945)
0810 07:19:48.364400 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.364405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.364415 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:48.364441 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.364443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.364452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.364522 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.365520 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.365529 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.365530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.365534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.365832 (+ 298us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.365869 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.365939 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.366272 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.366292 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260810 07:19:48.366490 23157 tablet_service.cc:1467] Processing CreateTablet for tablet be1b37988a61436db22b3da72de53b33 (DEFAULT_TABLE table=test-table [id=22033e15644d41e18e90b97644c630c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:48.366892 23157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be1b37988a61436db22b3da72de53b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:48.366880 22944 tablet_service.cc:1467] Processing CreateTablet for tablet be1b37988a61436db22b3da72de53b33 (DEFAULT_TABLE table=test-table [id=22033e15644d41e18e90b97644c630c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:48.366968 22723 tablet_service.cc:1467] Processing CreateTablet for tablet be1b37988a61436db22b3da72de53b33 (DEFAULT_TABLE table=test-table [id=22033e15644d41e18e90b97644c630c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:48.367233 22723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be1b37988a61436db22b3da72de53b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:48.367233 22944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be1b37988a61436db22b3da72de53b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:48.369977 23268 tablet_bootstrap.cc:492] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: Bootstrap starting.
I20260810 07:19:48.370388 23269 tablet_bootstrap.cc:492] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Bootstrap starting.
I20260810 07:19:48.370891 23268 tablet_bootstrap.cc:654] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:48.371096 23269 tablet_bootstrap.cc:654] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:48.371264 23268 log.cc:824] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:48.371433 23269 log.cc:824] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:48.371783 23271 tablet_bootstrap.cc:492] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Bootstrap starting.
I20260810 07:19:48.372116 23268 tablet_bootstrap.cc:492] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: No bootstrap required, opened a new log
I20260810 07:19:48.372206 23268 ts_tablet_manager.cc:1397] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:48.372242 23269 tablet_bootstrap.cc:492] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: No bootstrap required, opened a new log
I20260810 07:19:48.372310 23269 ts_tablet_manager.cc:1397] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:48.372490 23271 tablet_bootstrap.cc:654] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:48.372874 23271 log.cc:824] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:48.373788 23271 tablet_bootstrap.cc:492] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: No bootstrap required, opened a new log
I20260810 07:19:48.373857 23271 ts_tablet_manager.cc:1397] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:48.374641 23269 raft_consensus.cc:348] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.374769 23269 raft_consensus.cc:374] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:48.374790 23269 raft_consensus.cc:729] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39488c35c84846919641c5fc421f2480, State: Initialized, Role: FOLLOWER
I20260810 07:19:48.374926 23269 consensus_queue.cc:260] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.375012 23268 raft_consensus.cc:348] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.375181 23268 raft_consensus.cc:374] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:48.375196 23269 ts_tablet_manager.cc:1428] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:48.375226 23268 raft_consensus.cc:729] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76c89b7b7dc4265b0d7867ecef7203e, State: Initialized, Role: FOLLOWER
I20260810 07:19:48.375312 23050 heartbeater.cc:503] Master 127.14.164.62:46069 was elected leader, sending a full tablet report...
I20260810 07:19:48.375349 23268 consensus_queue.cc:260] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.375618 23268 ts_tablet_manager.cc:1428] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:48.375738 22839 heartbeater.cc:503] Master 127.14.164.62:46069 was elected leader, sending a full tablet report...
I20260810 07:19:48.376698 23271 raft_consensus.cc:348] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.376878 23271 raft_consensus.cc:374] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:48.376912 23271 raft_consensus.cc:729] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c0f85e948b4878a9280644707fba75, State: Initialized, Role: FOLLOWER
I20260810 07:19:48.377022 23271 consensus_queue.cc:260] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.377241 23271 ts_tablet_manager.cc:1428] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:48.377319 23250 heartbeater.cc:503] Master 127.14.164.62:46069 was elected leader, sending a full tablet report...
I20260810 07:19:48.431088 23279 raft_consensus.cc:482] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:48.431236 23279 raft_consensus.cc:504] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.431836 23279 leader_election.cc:288] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39488c35c84846919641c5fc421f2480 (127.14.164.2:46847), 61c0f85e948b4878a9280644707fba75 (127.14.164.3:41961)
I20260810 07:19:48.433171 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.431955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:46847 (local address 127.14.164.1:58001)
0810 07:19:48.432027 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.432040 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.432081 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.432210 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.432211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.432214 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.432843 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.432844 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.432852 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.432864 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.433043 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.433044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.433044 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.433088 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.433103 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 07:19:48.433174 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.432044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:58001 (local address 127.14.164.2:46847)
0810 07:19:48.432087 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.432089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.432116 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:19:48.432144 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.432145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.432150 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.432194 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.432951 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.432952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.432953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.432958 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.432977 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.432982 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.433042 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.433099 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.433112 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:48.433521 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.432261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:39451 (local address 127.14.164.3:41961)
0810 07:19:48.432340 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.432342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.432352 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:48.432379 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.432380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.432385 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.432461 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.433137 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.433139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.433139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.433144 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.433162 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.433167 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.433279 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.433425 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.433444 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:48.433573 22980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1b37988a61436db22b3da72de53b33" candidate_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39488c35c84846919641c5fc421f2480" is_pre_election: true
I20260810 07:19:48.433714 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.432042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:41961 (local address 127.14.164.1:39451)
0810 07:19:48.432242 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.432254 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.432310 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.432495 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.432496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.432498 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.433023 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:48.433024 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.433036 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:48.433049 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.433311 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.433312 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.433313 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.433624 (+ 311us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.433639 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":134,"threads_started":1}
I20260810 07:19:48.433727 22980 raft_consensus.cc:2454] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76c89b7b7dc4265b0d7867ecef7203e in term 0.
I20260810 07:19:48.433995 22648 leader_election.cc:302] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39488c35c84846919641c5fc421f2480, f76c89b7b7dc4265b0d7867ecef7203e; no voters:
I20260810 07:19:48.434139 23279 raft_consensus.cc:2781] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:48.434195 23279 raft_consensus.cc:482] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:48.434139 23188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1b37988a61436db22b3da72de53b33" candidate_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61c0f85e948b4878a9280644707fba75" is_pre_election: true
I20260810 07:19:48.434219 23279 raft_consensus.cc:3037] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:48.434314 23188 raft_consensus.cc:2454] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76c89b7b7dc4265b0d7867ecef7203e in term 0.
I20260810 07:19:48.435217 23279 raft_consensus.cc:504] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.435362 23279 leader_election.cc:288] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [CANDIDATE]: Term 1 election: Requested vote from peers 39488c35c84846919641c5fc421f2480 (127.14.164.2:46847), 61c0f85e948b4878a9280644707fba75 (127.14.164.3:41961)
I20260810 07:19:48.435540 22980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1b37988a61436db22b3da72de53b33" candidate_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39488c35c84846919641c5fc421f2480"
I20260810 07:19:48.435544 23188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1b37988a61436db22b3da72de53b33" candidate_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61c0f85e948b4878a9280644707fba75"
I20260810 07:19:48.435611 22980 raft_consensus.cc:3037] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:48.435650 23188 raft_consensus.cc:3037] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:48.436422 22980 raft_consensus.cc:2454] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76c89b7b7dc4265b0d7867ecef7203e in term 1.
I20260810 07:19:48.436441 23188 raft_consensus.cc:2454] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76c89b7b7dc4265b0d7867ecef7203e in term 1.
I20260810 07:19:48.436597 22648 leader_election.cc:302] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39488c35c84846919641c5fc421f2480, f76c89b7b7dc4265b0d7867ecef7203e; no voters:
I20260810 07:19:48.436694 23279 raft_consensus.cc:2781] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:48.436895 23279 raft_consensus.cc:686] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 1 LEADER]: Becoming Leader. State: Replica: f76c89b7b7dc4265b0d7867ecef7203e, State: Running, Role: LEADER
I20260810 07:19:48.437018 23279 consensus_queue.cc:237] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } }
I20260810 07:19:48.437909 22528 catalog_manager.cc:5495] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e reported cstate change: term changed from 0 to 1, leader changed from <none> to f76c89b7b7dc4265b0d7867ecef7203e (127.14.164.1). New cstate: current_term: 1 leader_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f76c89b7b7dc4265b0d7867ecef7203e" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32945 } health_report { overall_health: HEALTHY } } }
I20260810 07:19:48.444416 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.443481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.14.164.1:32945)
0810 07:19:48.443544 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.443546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.443553 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:19:48.443584 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.443585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.443596 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.443650 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.444244 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.444245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.444245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.444248 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.444262 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.444265 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.444312 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.444355 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.444365 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:19:48.446576 23188 raft_consensus.cc:1261] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 1 FOLLOWER]: Refusing update from remote peer f76c89b7b7dc4265b0d7867ecef7203e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:48.446576 22980 raft_consensus.cc:1261] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 1 FOLLOWER]: Refusing update from remote peer f76c89b7b7dc4265b0d7867ecef7203e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:48.447000 23285 consensus_queue.cc:1035] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61c0f85e948b4878a9280644707fba75" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:48.447160 23279 consensus_queue.cc:1035] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39488c35c84846919641c5fc421f2480" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:48.450392 23295 mvcc.cc:205] Tried to move back new op lower bound from 7316874807073742848 to 7316874807038717952. Current Snapshot: MvccSnapshot[applied={T|T < 7316874807073742848 or (T in {7316874807073742848})}]
I20260810 07:19:48.452994 22527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:48.453095 22527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:48.453948 22527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d7c0241583f4a489fa4b84a6b8b18e1: Sending DeleteTablet for 3 replicas of tablet be1b37988a61436db22b3da72de53b33
I20260810 07:19:48.454349 23157 tablet_service.cc:1514] Processing DeleteTablet for tablet be1b37988a61436db22b3da72de53b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:48 UTC) from {username='slave'} at 127.0.0.1:47806
I20260810 07:19:48.454341 22723 tablet_service.cc:1514] Processing DeleteTablet for tablet be1b37988a61436db22b3da72de53b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:48 UTC) from {username='slave'} at 127.0.0.1:51632
I20260810 07:19:48.454341 22944 tablet_service.cc:1514] Processing DeleteTablet for tablet be1b37988a61436db22b3da72de53b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:48 UTC) from {username='slave'} at 127.0.0.1:42898
I20260810 07:19:48.454597 23310 tablet_replica.cc:330] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e: stopping tablet replica
I20260810 07:19:48.454711 23310 raft_consensus.cc:2229] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:48.454766 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 22626
I20260810 07:19:48.454854 23310 raft_consensus.cc:2258] T be1b37988a61436db22b3da72de53b33 P f76c89b7b7dc4265b0d7867ecef7203e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:48.455035 23313 tablet_replica.cc:330] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: stopping tablet replica
I20260810 07:19:48.455129 23313 raft_consensus.cc:2229] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:48.455215 23313 raft_consensus.cc:2258] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:48.455415 23312 tablet_replica.cc:330] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: stopping tablet replica
I20260810 07:19:48.455507 23312 raft_consensus.cc:2229] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:48.455587 23312 raft_consensus.cc:2258] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:48.459852 23312 ts_tablet_manager.cc:1905] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:48.461884 23312 ts_tablet_manager.cc:1918] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:48.461964 23312 log.cc:1196] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/be1b37988a61436db22b3da72de53b33
I20260810 07:19:48.462211 23312 ts_tablet_manager.cc:1939] T be1b37988a61436db22b3da72de53b33 P 61c0f85e948b4878a9280644707fba75: Deleting consensus metadata
I20260810 07:19:48.463472 22504 catalog_manager.cc:4841] TS 61c0f85e948b4878a9280644707fba75 (127.14.164.3:41961): tablet be1b37988a61436db22b3da72de53b33 (table test-table [id=22033e15644d41e18e90b97644c630c6]) successfully deleted
I20260810 07:19:48.465883 23313 ts_tablet_manager.cc:1905] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:48.467954 23313 ts_tablet_manager.cc:1918] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:48.468051 23313 log.cc:1196] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/be1b37988a61436db22b3da72de53b33
I20260810 07:19:48.468331 23313 ts_tablet_manager.cc:1939] T be1b37988a61436db22b3da72de53b33 P 39488c35c84846919641c5fc421f2480: Deleting consensus metadata
I20260810 07:19:48.468961 22498 catalog_manager.cc:4841] TS 39488c35c84846919641c5fc421f2480 (127.14.164.2:46847): tablet be1b37988a61436db22b3da72de53b33 (table test-table [id=22033e15644d41e18e90b97644c630c6]) successfully deleted
W20260810 07:19:48.470978 22504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:32945 (error 108)
I20260810 07:19:48.471289 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 22842
I20260810 07:19:48.478907 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.478691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32945 (local address 127.0.0.1:51650)
0810 07:19:48.478780 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.478850 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:32945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 07:19:48.479122 22504 catalog_manager.cc:4584] TS f76c89b7b7dc4265b0d7867ecef7203e (127.14.164.1:32945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be1b37988a61436db22b3da72de53b33: Network error: Client connection negotiation failed: client connection to 127.14.164.1:32945: connect: Connection refused (error 111)
I20260810 07:19:48.488338 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 23055
I20260810 07:19:48.502892 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 22476
2026-08-10T07:19:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (728 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 07:19:48.530275 14992 mini_oidc.cc:152] Starting JWKS server
I20260810 07:19:48.531401 14992 webserver.cc:460] Webserver started at https://127.0.0.1:37743/ using document root <none> and password file <none>
I20260810 07:19:48.531494 14992 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:19:48.531631 14992 webserver.cc:460] Webserver started at http://127.0.0.1:45945/ using document root <none> and password file <none>
2026-08-10T07:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:48Z Disabled control of system clock
I20260810 07:19:48.541679 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42765
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37743/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/testchainca.cert with env {}
W20260810 07:19:48.627435 23324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.627662 23324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.627688 23324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.628938 23324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:48.628983 23324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.628995 23324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.629009 23324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:48.629020 23324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:19:48.629033 23324 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:19:48.629045 23324 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37743/jwks.json
I20260810 07:19:48.630668 23324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37743/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:48.630882 23324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.631055 23324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.636365 23332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.636405 23336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.636404 23330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.636636 23324 server_base.cc:1034] running on GCE node
I20260810 07:19:48.637018 23324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.637302 23324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.638499 23324 hybrid_clock.cc:648] HybridClock initialized: now 1786346388638488 us; error 40 us; skew 500 ppm
I20260810 07:19:48.640010 23324 webserver.cc:460] Webserver started at http://127.14.164.62:40633/ using document root <none> and password file <none>
I20260810 07:19:48.640302 23324 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.640383 23324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.640545 23324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.641748 23324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "0c7b8f37dff048129673b8d51ba3088a"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "ac051a439f10401ac7fcdadc5771bab7"
server_key_iv: "0f60e05841c94a0cdce13f5c2c603273"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.642160 23324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "0c7b8f37dff048129673b8d51ba3088a"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "ac051a439f10401ac7fcdadc5771bab7"
server_key_iv: "0f60e05841c94a0cdce13f5c2c603273"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.643553 23324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:48.644371 23343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.644570 23324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:48.644687 23324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "0c7b8f37dff048129673b8d51ba3088a"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "ac051a439f10401ac7fcdadc5771bab7"
server_key_iv: "0f60e05841c94a0cdce13f5c2c603273"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.644798 23324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.667102 23324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.667480 23324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.667647 23324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.672153 23324 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42765
I20260810 07:19:48.672196 23438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42765 every 8 connection(s)
I20260810 07:19:48.672609 23324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:48.673722 23441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:48.676311 23441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a: Bootstrap starting.
I20260810 07:19:48.676925 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 23324
I20260810 07:19:48.677044 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:48.677148 23441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:48.677261 14992 external_mini_cluster.cc:1428] Setting key 862f3069b53a6a30edd6f0f67d5b909d
I20260810 07:19:48.677546 23441 log.cc:824] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:48.678421 23441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a: No bootstrap required, opened a new log
I20260810 07:19:48.681079 23441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7b8f37dff048129673b8d51ba3088a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42765 } }
I20260810 07:19:48.681222 23441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:48.681246 23441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7b8f37dff048129673b8d51ba3088a, State: Initialized, Role: FOLLOWER
I20260810 07:19:48.681388 23441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7b8f37dff048129673b8d51ba3088a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42765 } }
I20260810 07:19:48.681470 23441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:48.681533 23441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:48.681586 23441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:48.682371 23441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7b8f37dff048129673b8d51ba3088a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42765 } }
I20260810 07:19:48.682492 23441 leader_election.cc:302] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7b8f37dff048129673b8d51ba3088a; no voters:
I20260810 07:19:48.682664 23441 leader_election.cc:288] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:48.682760 23449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:48.682960 23441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:48.683001 23449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [term 1 LEADER]: Becoming Leader. State: Replica: 0c7b8f37dff048129673b8d51ba3088a, State: Running, Role: LEADER
I20260810 07:19:48.683235 23449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7b8f37dff048129673b8d51ba3088a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42765 } }
I20260810 07:19:48.684113 23450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7b8f37dff048129673b8d51ba3088a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7b8f37dff048129673b8d51ba3088a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42765 } } }
I20260810 07:19:48.684254 23450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:48.684434 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.677894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.14.164.62:42765)
0810 07:19:48.678144 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.678153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.678187 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:19:48.678817 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.678827 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.678910 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.679065 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.679294 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.680771 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.681797 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.682442 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.682572 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.683482 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.683487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.683488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.683722 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.683774 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.683791 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.683863 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.684127 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.684216 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260810 07:19:48.684638 23451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7b8f37dff048129673b8d51ba3088a. Latest consensus state: current_term: 1 leader_uuid: "0c7b8f37dff048129673b8d51ba3088a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7b8f37dff048129673b8d51ba3088a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42765 } } }
I20260810 07:19:48.684861 23451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:48.684882 23475 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:48.684942 23475 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:48.685067 23476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:48.685221 23476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:48.685732 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:48.687167 23476 catalog_manager.cc:1347] Generated new cluster ID: f9d04dbfd8464b1ab213352e60a04fb0
I20260810 07:19:48.687229 23476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:48.692807 23476 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:48.693367 23476 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:48.696471 23476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c7b8f37dff048129673b8d51ba3088a: Generated new TSK 0
I20260810 07:19:48.696647 23476 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:19:48.722150 23487 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 07:19:48.722407 23487 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:19:48.722654 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.706196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.14.164.62:42765)
0810 07:19:48.706249 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.706251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.706406 (+ 155us) server_negotiation.cc:407] Connection header received
0810 07:19:48.706513 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.706513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.706524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.706585 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:19:48.706761 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.707247 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.708398 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.708590 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.708646 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.708903 (+ 257us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:19:48.722269 (+ 13366us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37743/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:19:48.722558 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45484: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37743/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 07:19:48.722750 23448 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45484: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37743/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 07:19:48.723220 23485 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:48.706093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42765 (local address 127.0.0.1:45484)
0810 07:19:48.706318 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.706331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.706415 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.706613 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.706614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.706616 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:19:48.706693 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.706695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.707341 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.707342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.708296 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.708298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.708670 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.708671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.708756 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.708782 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:19:48.722633 (+ 13851us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37743/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:19:48.722776 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.164.62:42765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37743/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.164.62:42765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37743/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 07:19:48.723917 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 23324
2026-08-10T07:19:48Z chronyd exiting
I20260810 07:19:48.770704 14992 test_util.cc:175] -----------------------------------------------
I20260810 07:19:48.770823 14992 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346363752364-14992-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 07:19:48.771891 14992 mini_oidc.cc:152] Starting JWKS server
I20260810 07:19:48.772698 14992 webserver.cc:460] Webserver started at https://127.0.0.1:38611/ using document root <none> and password file <none>
I20260810 07:19:48.772830 14992 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:19:48.773030 14992 webserver.cc:460] Webserver started at http://127.0.0.1:43749/ using document root <none> and password file <none>
2026-08-10T07:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:48Z Disabled control of system clock
I20260810 07:19:48.784067 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40245
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:40245
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38611/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/testchainca.cert with env {}
W20260810 07:19:48.870105 23497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.870335 23497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.870359 23497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.871640 23497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:48.871688 23497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.871701 23497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.871714 23497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:48.871726 23497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:19:48.871774 23497 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:19:48.871800 23497 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38611/jwks.json
I20260810 07:19:48.873500 23497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:40245
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38611/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:48.873781 23497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.873981 23497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.879386 23503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.879366 23504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.879401 23497 server_base.cc:1034] running on GCE node
W20260810 07:19:48.879354 23507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.879971 23497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.880301 23497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.881494 23497 hybrid_clock.cc:648] HybridClock initialized: now 1786346388881455 us; error 54 us; skew 500 ppm
I20260810 07:19:48.883235 23497 webserver.cc:460] Webserver started at http://127.14.164.62:38301/ using document root <none> and password file <none>
I20260810 07:19:48.883553 23497 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.883608 23497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.883754 23497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.884999 23497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "d213d66cc1e14f47b1be5c3f9c6021ab"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "fc38b39633b5f3abc437a27dd27ed5c8"
server_key_iv: "5e111dc0569407ade01facf4b7dd8760"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.885396 23497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "d213d66cc1e14f47b1be5c3f9c6021ab"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "fc38b39633b5f3abc437a27dd27ed5c8"
server_key_iv: "5e111dc0569407ade01facf4b7dd8760"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.886864 23497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:48.887825 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.888026 23497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:48.888147 23497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "d213d66cc1e14f47b1be5c3f9c6021ab"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-x5fp"
server_key: "fc38b39633b5f3abc437a27dd27ed5c8"
server_key_iv: "5e111dc0569407ade01facf4b7dd8760"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.888232 23497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.899923 23497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.900295 23497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.900470 23497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.904505 23497 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:40245
I20260810 07:19:48.904587 23603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:40245 every 8 connection(s)
I20260810 07:19:48.905001 23497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:48.905799 23604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:48.908859 23604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab: Bootstrap starting.
I20260810 07:19:48.909475 23604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:48.909693 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 23497
I20260810 07:19:48.909819 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:48.909801 23604 log.cc:824] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:48.910030 14992 external_mini_cluster.cc:1428] Setting key d61299bc199fd981ee1d8857f854ffe2
I20260810 07:19:48.910720 23604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab: No bootstrap required, opened a new log
I20260810 07:19:48.913619 23604 raft_consensus.cc:348] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40245 } }
I20260810 07:19:48.913806 23604 raft_consensus.cc:374] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:48.913838 23604 raft_consensus.cc:729] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d213d66cc1e14f47b1be5c3f9c6021ab, State: Initialized, Role: FOLLOWER
I20260810 07:19:48.913957 23604 consensus_queue.cc:260] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40245 } }
I20260810 07:19:48.914052 23604 raft_consensus.cc:388] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:48.914091 23604 raft_consensus.cc:482] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:48.914143 23604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:48.914755 23604 raft_consensus.cc:504] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40245 } }
I20260810 07:19:48.914896 23604 leader_election.cc:302] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d213d66cc1e14f47b1be5c3f9c6021ab; no voters:
I20260810 07:19:48.915058 23604 leader_election.cc:288] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:48.915131 23609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:48.915282 23609 raft_consensus.cc:686] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [term 1 LEADER]: Becoming Leader. State: Replica: d213d66cc1e14f47b1be5c3f9c6021ab, State: Running, Role: LEADER
I20260810 07:19:48.915329 23604 sys_catalog.cc:564] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:48.915397 23609 consensus_queue.cc:237] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40245 } }
I20260810 07:19:48.915724 23609 sys_catalog.cc:455] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader d213d66cc1e14f47b1be5c3f9c6021ab. Latest consensus state: current_term: 1 leader_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40245 } } }
I20260810 07:19:48.915824 23609 sys_catalog.cc:458] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:48.915733 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d213d66cc1e14f47b1be5c3f9c6021ab" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40245 } } }
I20260810 07:19:48.915908 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:48.916692 23620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:48.917243 23620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:48.917412 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.910562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.14.164.62:40245)
0810 07:19:48.910807 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.910816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.910864 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:19:48.912023 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.912033 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.912117 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.912278 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.912485 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.913950 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.914842 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.915490 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.916008 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.916439 (+ 431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.916444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.916444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:48.916677 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.916755 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:48.916772 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.917055 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.917095 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.917174 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260810 07:19:48.918352 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:48.918877 23620 catalog_manager.cc:1347] Generated new cluster ID: a05121fcff4441a5b6ee9bb080c57ead
I20260810 07:19:48.918932 23620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:48.928829 23620 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:48.929351 23620 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:48.932708 23620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d213d66cc1e14f47b1be5c3f9c6021ab: Generated new TSK 0
I20260810 07:19:48.932902 23620 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:19:48.963635 23655 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 07:19:48.963739 23655 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:19:48.964042 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.941930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.14.164.62:40245)
0810 07:19:48.942003 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.942006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.942122 (+ 116us) server_negotiation.cc:407] Connection header received
0810 07:19:48.942204 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.942205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.942219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.942310 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:19:48.942461 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.942962 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.943734 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.943961 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.944038 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.944227 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:19:48.963806 (+ 19579us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:19:48.963981 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 07:19:48.964120 23608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 07:19:48.964176 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:48.941849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40245 (local address 127.0.0.1:56388)
0810 07:19:48.942049 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.942065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.942142 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.942308 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.942309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.942311 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:19:48.942373 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.942375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.943047 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.943048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.943644 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.943646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.944039 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.944040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.944108 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.944116 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:19:48.963944 (+ 19828us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:19:48.964072 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.164.62:40245: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":121,"threads_started":1}
I20260810 07:19:48.964887 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 23497
2026-08-10T07:19:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 07:19:49.012152 14992 mini_oidc.cc:152] Starting JWKS server
I20260810 07:19:49.013092 14992 webserver.cc:460] Webserver started at https://127.0.0.1:46179/ using document root <none> and password file <none>
I20260810 07:19:49.013147 14992 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:19:49.013473 14992 webserver.cc:460] Webserver started at http://127.0.0.1:43983/ using document root <none> and password file <none>
2026-08-10T07:19:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:49Z Disabled control of system clock
I20260810 07:19:49.024044 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:41087
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:36497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:41087
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46179/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:19:49.112962 23662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:49.113193 23662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:49.113215 23662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:49.114332 23662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:49.114369 23662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:49.114382 23662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:49.114396 23662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:49.114408 23662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:19:49.114420 23662 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:19:49.114432 23662 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46179/jwks.json
I20260810 07:19:49.115770 23662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:36497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:41087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46179/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:49.115993 23662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:49.116250 23662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:49.122287 23671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:49.122366 23673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:49.122301 23670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:49.122717 23662 server_base.cc:1034] running on GCE node
I20260810 07:19:49.123035 23662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:49.123319 23662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:49.124480 23662 hybrid_clock.cc:648] HybridClock initialized: now 1786346389124453 us; error 39 us; skew 500 ppm
I20260810 07:19:49.125998 23662 webserver.cc:460] Webserver started at http://127.14.164.62:34419/ using document root <none> and password file <none>
I20260810 07:19:49.126296 23662 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:49.126377 23662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:49.126544 23662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:49.127708 23662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20"
format_stamp: "Formatted at 2026-08-10 07:19:49 on dist-test-slave-x5fp"
server_key: "e7e3bef728777745fda306e9354ff598"
server_key_iv: "b66a59b06c97e45ff7097d3b0bcd6095"
server_key_version: "encryptionkey@0"
I20260810 07:19:49.128108 23662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20"
format_stamp: "Formatted at 2026-08-10 07:19:49 on dist-test-slave-x5fp"
server_key: "e7e3bef728777745fda306e9354ff598"
server_key_iv: "b66a59b06c97e45ff7097d3b0bcd6095"
server_key_version: "encryptionkey@0"
I20260810 07:19:49.129688 23662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:49.130715 23683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:49.131024 23662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:49.131162 23662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20"
format_stamp: "Formatted at 2026-08-10 07:19:49 on dist-test-slave-x5fp"
server_key: "e7e3bef728777745fda306e9354ff598"
server_key_iv: "b66a59b06c97e45ff7097d3b0bcd6095"
server_key_version: "encryptionkey@0"
I20260810 07:19:49.131249 23662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:49.157816 23662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:49.158202 23662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:49.158368 23662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:49.162676 23662 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:41087
I20260810 07:19:49.162786 23765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:41087 every 8 connection(s)
I20260810 07:19:49.163137 23662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:49.163968 23768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:49.167349 23768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20: Bootstrap starting.
I20260810 07:19:49.168047 23768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:49.168380 23768 log.cc:824] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:49.169234 23768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20: No bootstrap required, opened a new log
I20260810 07:19:49.170136 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 23662
I20260810 07:19:49.170272 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:49.170465 14992 external_mini_cluster.cc:1428] Setting key cdc994dd025d5d6fd7892cc31f65dfb2
I20260810 07:19:49.171918 23768 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41087 } }
I20260810 07:19:49.172166 23768 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:49.172236 23768 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f902b45d6a4cddbcb0bc14aa20cf20, State: Initialized, Role: FOLLOWER
I20260810 07:19:49.172447 23768 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41087 } }
I20260810 07:19:49.172550 23768 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:49.172600 23768 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:49.172662 23768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:49.173384 23768 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41087 } }
I20260810 07:19:49.173519 23768 leader_election.cc:302] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f902b45d6a4cddbcb0bc14aa20cf20; no voters:
I20260810 07:19:49.173755 23768 leader_election.cc:288] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:49.173791 23774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:49.173918 23774 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [term 1 LEADER]: Becoming Leader. State: Replica: a0f902b45d6a4cddbcb0bc14aa20cf20, State: Running, Role: LEADER
I20260810 07:19:49.174010 23774 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41087 } }
I20260810 07:19:49.174081 23768 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:49.174404 23778 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0f902b45d6a4cddbcb0bc14aa20cf20. Latest consensus state: current_term: 1 leader_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41087 } } }
I20260810 07:19:49.174497 23778 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:49.174672 23775 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f902b45d6a4cddbcb0bc14aa20cf20" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41087 } } }
I20260810 07:19:49.174729 23775 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:49.174860 23782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:49.175499 23782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:49.177349 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.171038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.14.164.62:41087)
0810 07:19:49.171279 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.171287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.171314 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:19:49.171805 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.171814 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.171877 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.171998 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:49.172365 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.173795 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.174966 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.175512 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.176052 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.176477 (+ 425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:49.176481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:49.176482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:49.176761 (+ 279us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:49.176820 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:49.176833 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:49.176921 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:49.177014 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:19:49.177118 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260810 07:19:49.178439 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:49.181600 23782 catalog_manager.cc:1347] Generated new cluster ID: 51464b56ca7749da98890eed2a2f741c
I20260810 07:19:49.181661 23782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:49.192556 23782 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:49.193367 23782 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:49.207875 23782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0f902b45d6a4cddbcb0bc14aa20cf20: Generated new TSK 0
I20260810 07:19:49.208117 23782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:49.219718 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.215824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.14.164.62:41087)
0810 07:19:49.215902 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.215905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.216057 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:19:49.216114 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.216115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.216129 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.216209 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:19:49.216352 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.216947 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.218109 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.218307 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.218373 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.218592 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:19:49.219485 (+ 893us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:19:49.219641 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 07:19:49.219785 23773 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 07:19:49.219794 23812 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:49.215769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41087 (local address 127.0.0.1:32808)
0810 07:19:49.215968 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:49.215980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:49.216050 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:49.216216 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:49.216217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:49.216219 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:19:49.216275 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:49.216278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.217039 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.217040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:49.218006 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:49.218008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.218396 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.218398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:49.218458 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.218464 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:19:49.219594 (+ 1130us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:19:49.219689 (+ 95us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.164.62:41087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":132,"threads_started":1}
I20260810 07:19:49.220386 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 23662
E20260810 07:19:49.234218 23814 webserver.cc:533] Webserver: sslize failed:
E20260810 07:19:49.234301 23814 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T07:19:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T07:19:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:49Z Disabled control of system clock
I20260810 07:19:49.272593 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40133
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:38323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:40133
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/testchainca.cert with env {}
W20260810 07:19:49.357453 23818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:49.357671 23818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:49.357693 23818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:49.358827 23818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:49.358868 23818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:49.358881 23818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:49.358894 23818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:49.358906 23818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:19:49.358918 23818 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:19:49.358930 23818 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 07:19:49.360448 23818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:40133
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:49.360652 23818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:49.360903 23818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:49.366112 23831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:49.366106 23828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:49.366310 23818 server_base.cc:1034] running on GCE node
W20260810 07:19:49.366106 23829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:49.366739 23818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:49.367110 23818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:49.368286 23818 hybrid_clock.cc:648] HybridClock initialized: now 1786346389368269 us; error 62 us; skew 500 ppm
I20260810 07:19:49.369729 23818 webserver.cc:460] Webserver started at http://127.14.164.62:33425/ using document root <none> and password file <none>
I20260810 07:19:49.370038 23818 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:49.370118 23818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:49.370270 23818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:49.371392 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "10be5034f7c6494f8c8fda745350a8d9"
format_stamp: "Formatted at 2026-08-10 07:19:49 on dist-test-slave-x5fp"
server_key: "67aa8365d37f1d7a7f2fa01f3ee68079"
server_key_iv: "28549682a20b8b65cfe2c3bd0654030a"
server_key_version: "encryptionkey@0"
I20260810 07:19:49.371783 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "10be5034f7c6494f8c8fda745350a8d9"
format_stamp: "Formatted at 2026-08-10 07:19:49 on dist-test-slave-x5fp"
server_key: "67aa8365d37f1d7a7f2fa01f3ee68079"
server_key_iv: "28549682a20b8b65cfe2c3bd0654030a"
server_key_version: "encryptionkey@0"
I20260810 07:19:49.373454 23818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260810 07:19:49.374295 23841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:49.374498 23818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:49.374614 23818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "10be5034f7c6494f8c8fda745350a8d9"
format_stamp: "Formatted at 2026-08-10 07:19:49 on dist-test-slave-x5fp"
server_key: "67aa8365d37f1d7a7f2fa01f3ee68079"
server_key_iv: "28549682a20b8b65cfe2c3bd0654030a"
server_key_version: "encryptionkey@0"
I20260810 07:19:49.374694 23818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:49.397506 23818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:49.397881 23818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:49.398061 23818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:49.402529 23818 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:40133
I20260810 07:19:49.402578 23915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:40133 every 8 connection(s)
I20260810 07:19:49.402987 23818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:49.404079 23916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:49.406810 23916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9: Bootstrap starting.
I20260810 07:19:49.407480 23916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:49.407757 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 23818
I20260810 07:19:49.407825 23916 log.cc:824] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:49.407872 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:49.408088 14992 external_mini_cluster.cc:1428] Setting key 4d80a94ff955375055058a3514ccaa53
I20260810 07:19:49.408860 23916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9: No bootstrap required, opened a new log
I20260810 07:19:49.411900 23916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10be5034f7c6494f8c8fda745350a8d9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40133 } }
I20260810 07:19:49.412079 23916 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:49.412142 23916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10be5034f7c6494f8c8fda745350a8d9, State: Initialized, Role: FOLLOWER
I20260810 07:19:49.412282 23916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10be5034f7c6494f8c8fda745350a8d9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40133 } }
I20260810 07:19:49.412392 23916 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:49.412446 23916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:49.412499 23916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:49.413293 23916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10be5034f7c6494f8c8fda745350a8d9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40133 } }
I20260810 07:19:49.413416 23916 leader_election.cc:302] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10be5034f7c6494f8c8fda745350a8d9; no voters:
I20260810 07:19:49.413590 23916 leader_election.cc:288] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:49.413661 23925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:49.413877 23916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:49.414026 23925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [term 1 LEADER]: Becoming Leader. State: Replica: 10be5034f7c6494f8c8fda745350a8d9, State: Running, Role: LEADER
I20260810 07:19:49.414134 23925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10be5034f7c6494f8c8fda745350a8d9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40133 } }
I20260810 07:19:49.414510 23925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10be5034f7c6494f8c8fda745350a8d9. Latest consensus state: current_term: 1 leader_uuid: "10be5034f7c6494f8c8fda745350a8d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10be5034f7c6494f8c8fda745350a8d9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40133 } } }
I20260810 07:19:49.414592 23925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:49.414919 23926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10be5034f7c6494f8c8fda745350a8d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10be5034f7c6494f8c8fda745350a8d9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40133 } } }
I20260810 07:19:49.415006 23926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:49.415174 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.408623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.14.164.62:40133)
0810 07:19:49.408878 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.408888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.408925 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:19:49.409508 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.409518 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.409604 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.409771 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:49.409942 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.411555 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.412385 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.413107 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.413253 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.414015 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:49.414020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:49.414020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:49.414259 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:49.414313 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:49.414328 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:49.414408 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:49.414701 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 07:19:49.414789 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
I20260810 07:19:49.415385 23936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:49.416035 23936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:49.416445 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:49.417831 23936 catalog_manager.cc:1347] Generated new cluster ID: f59d1984b3f64927a46c286f920fedb8
I20260810 07:19:49.417886 23936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:49.427712 23936 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:49.428421 23936 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:49.431192 23936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10be5034f7c6494f8c8fda745350a8d9: Generated new TSK 0
I20260810 07:19:49.431376 23936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:49.523398 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.438406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.14.164.62:40133)
0810 07:19:49.438474 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.438476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.438583 (+ 107us) server_negotiation.cc:407] Connection header received
0810 07:19:49.438678 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.438679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.438697 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.438778 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:19:49.438963 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.439465 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.440514 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.440767 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.440844 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.441059 (+ 215us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:19:49.523078 (+ 82019us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:19:49.523304 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42088: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:19:49.523501 23962 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:49.438328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40133 (local address 127.0.0.1:42088)
0810 07:19:49.438510 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:49.438521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:49.438603 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:49.438778 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:49.438779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:49.438781 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:19:49.438845 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:49.438848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.439802 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.439803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:49.440422 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:49.440424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.440865 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.440867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:49.440942 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.440947 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:19:49.523239 (+ 82292us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:19:49.523376 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.164.62:40133: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
W20260810 07:19:49.523494 23924 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42088: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260810 07:19:49.523959 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 23818
2026-08-10T07:19:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (287 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (98 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:49 dist-test-slave-x5fp krb5kdc[23982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:49 dist-test-slave-x5fp krb5kdc[23982](info): set up 2 sockets
Aug 10 07:19:49 dist-test-slave-x5fp krb5kdc[23982](info): commencing operation
krb5kdc: starting...
W20260810 07:19:50.997359 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:19:51.140823 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40999
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:51.226241 24003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:51.226508 24003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:51.226552 24003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.227795 24003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:51.227873 24003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.227897 24003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.227919 24003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:51.227948 24003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:51.229501 24003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:51.229792 24003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.230041 24003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.235100 24015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.235114 24012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.235128 24011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.235386 24003 server_base.cc:1034] running on GCE node
I20260810 07:19:51.235672 24003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.235929 24003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.237105 24003 hybrid_clock.cc:648] HybridClock initialized: now 1786346391237091 us; error 28 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.239830 24003 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:19:51.240339 24003 webserver.cc:460] Webserver started at http://127.14.164.62:40891/ using document root <none> and password file <none>
I20260810 07:19:51.240604 24003 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.240653 24003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.240824 24003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.241930 24003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "81feefb375004bb79d0a1c4fba973dfc"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "fd3202ed1793c1257ac17b2941f3536e"
server_key_iv: "dee59029712787d5b0fcbaae754184ad"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.242307 24003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "81feefb375004bb79d0a1c4fba973dfc"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "fd3202ed1793c1257ac17b2941f3536e"
server_key_iv: "dee59029712787d5b0fcbaae754184ad"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.243734 24003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:51.244596 24022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.244822 24003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:51.244939 24003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "81feefb375004bb79d0a1c4fba973dfc"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "fd3202ed1793c1257ac17b2941f3536e"
server_key_iv: "dee59029712787d5b0fcbaae754184ad"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.245033 24003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.258843 24003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.261334 24003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.261515 24003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.265663 24003 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:40999
I20260810 07:19:51.265712 24098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:40999 every 8 connection(s)
I20260810 07:19:51.266095 24003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:51.267124 24099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:51.267143 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 24003
I20260810 07:19:51.267239 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:51.267441 14992 external_mini_cluster.cc:1428] Setting key d71828c73db9eb0f50eb51036bd97944
I20260810 07:19:51.270761 24099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc: Bootstrap starting.
I20260810 07:19:51.271535 24099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:51.271848 24099 log.cc:824] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:51.272651 24099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc: No bootstrap required, opened a new log
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:51.275313 24099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81feefb375004bb79d0a1c4fba973dfc" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40999 } }
I20260810 07:19:51.275491 24099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:51.275578 24099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81feefb375004bb79d0a1c4fba973dfc, State: Initialized, Role: FOLLOWER
I20260810 07:19:51.275719 24099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81feefb375004bb79d0a1c4fba973dfc" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40999 } }
I20260810 07:19:51.275806 24099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:51.275870 24099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:51.275930 24099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:51.276779 24099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81feefb375004bb79d0a1c4fba973dfc" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40999 } }
I20260810 07:19:51.276898 24099 leader_election.cc:302] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81feefb375004bb79d0a1c4fba973dfc; no voters:
I20260810 07:19:51.277076 24099 leader_election.cc:288] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:51.277133 24105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:51.277359 24099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:51.277343 24105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [term 1 LEADER]: Becoming Leader. State: Replica: 81feefb375004bb79d0a1c4fba973dfc, State: Running, Role: LEADER
I20260810 07:19:51.278163 24105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81feefb375004bb79d0a1c4fba973dfc" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40999 } }
I20260810 07:19:51.278813 24108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81feefb375004bb79d0a1c4fba973dfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81feefb375004bb79d0a1c4fba973dfc" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40999 } } }
I20260810 07:19:51.278916 24108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:51.279289 24106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81feefb375004bb79d0a1c4fba973dfc. Latest consensus state: current_term: 1 leader_uuid: "81feefb375004bb79d0a1c4fba973dfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81feefb375004bb79d0a1c4fba973dfc" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40999 } } }
I20260810 07:19:51.279510 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.268037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35524 (local address 127.14.164.62:40999)
0810 07:19:51.268241 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.268250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.268276 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:19:51.268839 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.268847 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.268898 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.269024 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.269851 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.270998 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.272114 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.272644 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.272814 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.275468 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.275471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.275472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.275482 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.277626 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.278057 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.278061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.278061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.278175 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.278403 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.278404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.278404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.278741 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.278901 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.279067 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.279228 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260810 07:19:51.279562 24106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:51.279742 24129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:51.279779 24129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:51.279850 24130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:51.280063 24130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:51.281635 24130 catalog_manager.cc:1347] Generated new cluster ID: 11390e7d617b4675ad96a217eb0aa2ff
I20260810 07:19:51.281697 24130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:51.323196 24130 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:51.323983 24130 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:51.330484 24130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81feefb375004bb79d0a1c4fba973dfc: Generated new TSK 0
I20260810 07:19:51.330714 24130 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:19:51.372357 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40999
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:51.459460 24139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:51.459681 24139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:51.459704 24139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.460844 24139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.460882 24139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.460909 24139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:51.462402 24139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:51.462600 24139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.462800 24139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.468209 24152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.468225 24150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.468449 24139 server_base.cc:1034] running on GCE node
W20260810 07:19:51.468207 24149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.468881 24139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.469180 24139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.470376 24139 hybrid_clock.cc:648] HybridClock initialized: now 1786346391470349 us; error 36 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.473479 24139 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:19:51.473970 24139 webserver.cc:460] Webserver started at http://127.14.164.1:41225/ using document root <none> and password file <none>
I20260810 07:19:51.474229 24139 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.474282 24139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.474438 24139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.475648 24139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "f1b5fe9aa2ab4b63bdff522310efbcf4"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "ebd8692cc6fb01c3c7fc745a3487e960"
server_key_iv: "d0a90d8637ab36b241d60ea6c4a05102"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.476016 24139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "f1b5fe9aa2ab4b63bdff522310efbcf4"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "ebd8692cc6fb01c3c7fc745a3487e960"
server_key_iv: "d0a90d8637ab36b241d60ea6c4a05102"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.477705 24139 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:51.478700 24160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.478986 24139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:51.479115 24139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "f1b5fe9aa2ab4b63bdff522310efbcf4"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "ebd8692cc6fb01c3c7fc745a3487e960"
server_key_iv: "d0a90d8637ab36b241d60ea6c4a05102"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.479205 24139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.505093 24139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.507929 24139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.508117 24139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.508430 24139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:51.508860 24139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:51.508904 24139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.508930 24139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:51.508945 24139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.515841 24139 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:38665
I20260810 07:19:51.515980 24325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:38665 every 8 connection(s)
I20260810 07:19:51.516331 24139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:51.518873 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 24139
I20260810 07:19:51.519022 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:51.519244 14992 external_mini_cluster.cc:1428] Setting key c1f24306ecd12be9edd65e701eadc34a
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:51.527141 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.517259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:41645 (local address 127.14.164.62:40999)
0810 07:19:51.517353 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.517355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.517588 (+ 233us) server_negotiation.cc:407] Connection header received
0810 07:19:51.518558 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.518560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.518572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.518703 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.519606 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.520133 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.521359 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.521590 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.521679 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.524055 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.524056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.524057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.524064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.525610 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.526091 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.526092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.526093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.526157 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.526432 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.526433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.526433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.526563 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.526643 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.526950 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.527043 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:19:51.527377 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.517224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40999 (local address 127.14.164.1:41645)
0810 07:19:51.517426 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.517473 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.518420 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.518804 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.518808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.519219 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:51.519490 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.519496 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.520240 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.520241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.521263 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.521266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.521693 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.521694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.521836 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.522433 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:51.522443 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:51.523927 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:51.525718 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.525726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.525729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.525990 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.526237 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.526238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.526238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.526359 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.526635 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:51.526642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:51.526795 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.526926 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.527102 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260810 07:19:51.527992 24326 heartbeater.cc:347] Connected to a master server at 127.14.164.62:40999
I20260810 07:19:51.528121 24326 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:51.528368 24326 heartbeater.cc:511] Master 127.14.164.62:40999 requested a full tablet report, sending...
I20260810 07:19:51.529019 24050 ts_manager.cc:194] Registered new tserver with Master: f1b5fe9aa2ab4b63bdff522310efbcf4 (127.14.164.1:38665)
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
I20260810 07:19:51.529733 24050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:41645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:19:51.553440 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40999
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:51.640239 24333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:51.640463 24333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:51.640487 24333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.641772 24333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.641824 24333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.641851 24333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:51.643420 24333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:51.643682 24333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.643940 24333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.649179 24340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.649185 24344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.649210 24338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.649740 24333 server_base.cc:1034] running on GCE node
I20260810 07:19:51.650012 24333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.650303 24333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.651475 24333 hybrid_clock.cc:648] HybridClock initialized: now 1786346391651460 us; error 35 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.654320 24333 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:19:51.654831 24333 webserver.cc:460] Webserver started at http://127.14.164.2:37749/ using document root <none> and password file <none>
I20260810 07:19:51.655125 24333 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.655174 24333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.655328 24333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.656445 24333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "8875f2aadc6f43b8a9160fc93ca587c3"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "c6c0d9ff6746c12e5b474f3b1dea1678"
server_key_iv: "54f2d8e019a38dd684a82cab72280198"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.656869 24333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "8875f2aadc6f43b8a9160fc93ca587c3"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "c6c0d9ff6746c12e5b474f3b1dea1678"
server_key_iv: "54f2d8e019a38dd684a82cab72280198"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.659587 24333 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:19:51.660454 24354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.660815 24333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:51.660970 24333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "8875f2aadc6f43b8a9160fc93ca587c3"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "c6c0d9ff6746c12e5b474f3b1dea1678"
server_key_iv: "54f2d8e019a38dd684a82cab72280198"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.661067 24333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.673836 24333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.676244 24333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.676432 24333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.676710 24333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:51.677182 24333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:51.677248 24333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.677312 24333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:51.677350 24333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.683606 24333 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:33319
I20260810 07:19:51.683676 24519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:33319 every 8 connection(s)
I20260810 07:19:51.684064 24333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:51.689811 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 24333
I20260810 07:19:51.689967 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:51.690202 14992 external_mini_cluster.cc:1428] Setting key eceaf3d54d6ceb04716d651137c03c52
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:51.694553 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.684927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:38385 (local address 127.14.164.62:40999)
0810 07:19:51.685003 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.685005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.685243 (+ 238us) server_negotiation.cc:407] Connection header received
0810 07:19:51.686213 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.686215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.686225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.686282 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.687015 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.687526 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.688539 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.688777 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.688847 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.691453 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.691454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.691455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.691461 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.693049 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.693470 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.693471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.693472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.693544 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.693795 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.693796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.693796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.693985 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.694061 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.694317 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.694414 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:51.694576 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.684937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40999 (local address 127.14.164.2:38385)
0810 07:19:51.685104 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.685144 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.686105 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.686401 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.686410 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.686724 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:51.686929 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.686934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.687613 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.687614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.688454 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.688456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.688863 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.688864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.688975 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.689561 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:51.689570 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:51.691342 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:51.693148 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.693155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.693157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.693396 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.693616 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.693616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.693617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.693728 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.694058 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:51.694064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:51.694182 (+ 118us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.694268 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.694404 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260810 07:19:51.695117 24520 heartbeater.cc:347] Connected to a master server at 127.14.164.62:40999
I20260810 07:19:51.695242 24520 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:51.695443 24520 heartbeater.cc:511] Master 127.14.164.62:40999 requested a full tablet report, sending...
I20260810 07:19:51.695888 24050 ts_manager.cc:194] Registered new tserver with Master: 8875f2aadc6f43b8a9160fc93ca587c3 (127.14.164.2:33319)
I20260810 07:19:51.696343 24050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:38385
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:19:51.721704 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40999
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:51.808171 24530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:51.808426 24530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:51.808470 24530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.809648 24530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.809718 24530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.809763 24530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:51.811247 24530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:51.811527 24530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.811765 24530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.817279 24544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.817279 24539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.817312 24530 server_base.cc:1034] running on GCE node
W20260810 07:19:51.817308 24538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.817754 24530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.818032 24530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.819223 24530 hybrid_clock.cc:648] HybridClock initialized: now 1786346391819211 us; error 39 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.822070 24530 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:19:51.822623 24530 webserver.cc:460] Webserver started at http://127.14.164.3:43717/ using document root <none> and password file <none>
I20260810 07:19:51.822866 24530 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.822939 24530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.823100 24530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.824198 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "fb6e85f4c73f4460acb727a4ca04b986"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "dd4eee998b70166a2a77d6aa3eac6d26"
server_key_iv: "40470a832efa523e78a3985564014fb9"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.824582 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "fb6e85f4c73f4460acb727a4ca04b986"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "dd4eee998b70166a2a77d6aa3eac6d26"
server_key_iv: "40470a832efa523e78a3985564014fb9"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.826081 24530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:51.826911 24554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.827119 24530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:51.827243 24530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "fb6e85f4c73f4460acb727a4ca04b986"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-x5fp"
server_key: "dd4eee998b70166a2a77d6aa3eac6d26"
server_key_iv: "40470a832efa523e78a3985564014fb9"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.827325 24530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.844597 24530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.846940 24530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.847123 24530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.847411 24530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:51.847752 24530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:51.847787 24530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.847810 24530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:51.847822 24530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.853977 24530 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:43781
I20260810 07:19:51.854049 24722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:43781 every 8 connection(s)
I20260810 07:19:51.854358 24530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:51.858227 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 24530
I20260810 07:19:51.858358 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:51.858573 14992 external_mini_cluster.cc:1428] Setting key f764c4b3a15a3c40005dfc801486470c
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:19:51.865193 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.855110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:49961 (local address 127.14.164.62:40999)
0810 07:19:51.855180 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.855182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.855433 (+ 251us) server_negotiation.cc:407] Connection header received
0810 07:19:51.856370 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.856371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.856381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.856442 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.857186 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.857688 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.859431 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.859637 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.859731 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.862337 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.862338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.862338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.862345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.863653 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.864194 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.864195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.864195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.864253 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.864480 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.864480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.864480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.864605 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.864657 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.865010 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.865118 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:19:51.865384 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.855094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40999 (local address 127.14.164.3:49961)
0810 07:19:51.855295 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.855333 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.856264 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.856519 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.856522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.856866 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:51.857072 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.857078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.857807 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.857808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.859283 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.859287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.859740 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.859741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.859908 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.860553 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:51.860562 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:51.862240 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:51.863758 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.863765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.863767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.864124 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.864318 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.864318 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.864318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.864421 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.864670 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:51.864677 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:51.864885 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.864980 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.865156 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260810 07:19:51.865865 24723 heartbeater.cc:347] Connected to a master server at 127.14.164.62:40999
I20260810 07:19:51.865978 24723 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:51.866163 24723 heartbeater.cc:511] Master 127.14.164.62:40999 requested a full tablet report, sending...
I20260810 07:19:51.866567 24050 ts_manager.cc:194] Registered new tserver with Master: fb6e85f4c73f4460acb727a4ca04b986 (127.14.164.3:43781)
I20260810 07:19:51.866953 24050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:49961
I20260810 07:19:51.870704 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/test-admin.keytab.
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.890978 14992 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 07:19:51.893044 24732 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:51.892097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.14.164.1:38665)
0810 07:19:51.892328 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.892333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.892344 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:51.892373 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.892375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.892390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.892478 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.892930 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56514: BlockingRecv error: recv got EOF from 127.0.0.1:56514 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
W20260810 07:19:51.893230 14992 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.894335 14992 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.895243 14992 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:51.904193 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.895826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.14.164.1:38665)
0810 07:19:51.895896 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.895898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.895992 (+ 94us) server_negotiation.cc:407] Connection header received
0810 07:19:51.896023 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.896024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.896036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.896093 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.896555 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.897401 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.898252 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.898632 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.898728 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.901037 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.901046 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.901046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.901054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.902778 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.903130 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.903133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.903133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.903194 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.903479 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.903480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.903481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.903656 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.903803 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.903941 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.904070 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:19:51.906527 24732 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:51.905685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.14.164.1:38665)
0810 07:19:51.905751 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.905753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.905855 (+ 102us) server_negotiation.cc:407] Connection header received
0810 07:19:51.905947 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.905948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.905960 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.906006 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.906451 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56536: BlockingRecv error: recv got EOF from 127.0.0.1:56536 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:19:51.906682 14992 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.907702 14992 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.908571 14992 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:51.916919 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.909293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56552 (local address 127.14.164.1:38665)
0810 07:19:51.909361 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.909363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.909373 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:51.909441 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.909441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.909451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.909503 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.909908 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.910459 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.911191 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.911411 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.911482 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.913831 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.913832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.913832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.913841 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.915726 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.916102 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.916103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.916103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.916164 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.916386 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.916387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.916387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.916490 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.916569 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.916658 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.916850 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:19:51.918660 24732 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:51.918019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56556 (local address 127.14.164.1:38665)
0810 07:19:51.918076 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.918078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.918171 (+ 93us) server_negotiation.cc:407] Connection header received
0810 07:19:51.918252 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.918253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.918263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.918332 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.918618 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56556: BlockingRecv error: recv got EOF from 127.0.0.1:56556 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:19:51.918901 14992 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346363752364-14992-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.919917 14992 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.920841 14992 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:19:51 dist-test-slave-x5fp krb5kdc[23982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:19:51.928524 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.921430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.14.164.1:38665)
0810 07:19:51.921478 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.921480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.921513 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:51.921591 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.921592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.921601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.921673 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.922113 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.922599 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.923335 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.923521 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.923592 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.925868 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.925870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.925870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.925879 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.927347 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.927720 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.927721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.927721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.927785 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.927990 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.927991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.927992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.928098 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.928200 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.928338 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.928418 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:19:51.929245 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 24139
I20260810 07:19:51.939069 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 24333
I20260810 07:19:51.950256 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 24530
I20260810 07:19:51.964068 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 24003
2026-08-10T07:19:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2247 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:52 dist-test-slave-x5fp krb5kdc[24752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:52 dist-test-slave-x5fp krb5kdc[24752](info): set up 2 sockets
Aug 10 07:19:52 dist-test-slave-x5fp krb5kdc[24752](info): commencing operation
krb5kdc: starting...
W20260810 07:19:53.309552 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab.
Entry for principal oryx/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab.
I20260810 07:19:53.448982 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39011
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab
--principal=oryx/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:53.537007 24773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:53.537285 24773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:53.537328 24773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:53.538486 24773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:53.538563 24773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:53.538591 24773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:53.538623 24773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:53.538648 24773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:53.540202 24773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab
--principal=oryx/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:53.540506 24773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:53.540802 24773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:53.546002 24780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.545981 24787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.546262 24773 server_base.cc:1034] running on GCE node
W20260810 07:19:53.546172 24782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.546663 24773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:53.547032 24773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:53.548213 24773 hybrid_clock.cc:648] HybridClock initialized: now 1786346393548179 us; error 56 us; skew 500 ppm
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:53.557502 24773 init.cc:377] Logged in from keytab as oryx/127.14.164.62@KRBTEST.COM (short username oryx)
I20260810 07:19:53.558125 24773 webserver.cc:460] Webserver started at http://127.14.164.62:37941/ using document root <none> and password file <none>
I20260810 07:19:53.558440 24773 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:53.558527 24773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:53.558723 24773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:53.560015 24773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "b6e26534bc5a42bf8955212b6e8295c8"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "acc6015de0932e871a771179da46e19b"
server_key_iv: "8aa1cea288e13588f664372872c5f135"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.560483 24773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "b6e26534bc5a42bf8955212b6e8295c8"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "acc6015de0932e871a771179da46e19b"
server_key_iv: "8aa1cea288e13588f664372872c5f135"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.562280 24773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:53.563400 24796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.563650 24773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:53.563786 24773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "b6e26534bc5a42bf8955212b6e8295c8"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "acc6015de0932e871a771179da46e19b"
server_key_iv: "8aa1cea288e13588f664372872c5f135"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.563872 24773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:53.581594 24773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:53.583871 24773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:53.584052 24773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:53.588418 24773 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:39011
I20260810 07:19:53.588464 24876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:39011 every 8 connection(s)
I20260810 07:19:53.588893 24773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:53.590005 24877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:53.592550 24877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Bootstrap starting.
I20260810 07:19:53.593194 24877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:53.593513 24877 log.cc:824] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:53.594380 24877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: No bootstrap required, opened a new log
I20260810 07:19:53.595866 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 24773
I20260810 07:19:53.596007 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:19:53.596220 14992 external_mini_cluster.cc:1428] Setting key 86ec2b77cab904ad305d3b53f06ccbb1
I20260810 07:19:53.596179 24877 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } }
I20260810 07:19:53.596360 24877 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:53.596415 24877 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e26534bc5a42bf8955212b6e8295c8, State: Initialized, Role: FOLLOWER
I20260810 07:19:53.596580 24877 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } }
I20260810 07:19:53.596751 24877 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:53.596864 24877 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:53.596939 24877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:53.598213 24877 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } }
I20260810 07:19:53.598366 24877 leader_election.cc:302] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6e26534bc5a42bf8955212b6e8295c8; no voters:
I20260810 07:19:53.598554 24877 leader_election.cc:288] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:53.598596 24884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:53.598757 24884 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 LEADER]: Becoming Leader. State: Replica: b6e26534bc5a42bf8955212b6e8295c8, State: Running, Role: LEADER
I20260810 07:19:53.598855 24877 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:53.598932 24884 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } }
I20260810 07:19:53.599383 24884 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e26534bc5a42bf8955212b6e8295c8. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } }
I20260810 07:19:53.599390 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } }
I20260810 07:19:53.599594 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:53.599584 24884 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:53.599977 24904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:53.600888 24904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:53.602666 24904 catalog_manager.cc:1347] Generated new cluster ID: ef8aacfb45514870bf4025d483ab3a5f
I20260810 07:19:53.602728 24904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:53.608065 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.596863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.14.164.62:39011)
0810 07:19:53.597082 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.597092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.597127 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:19:53.597234 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.597239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.597298 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.597413 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.598490 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.599866 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.601040 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.601647 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.601778 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.604435 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.604440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.604440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.604455 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.606316 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.606722 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.606735 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.606735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.606852 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.607085 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.607086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.607087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.607376 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.607556 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.607634 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.607788 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260810 07:19:53.622063 24904 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:53.622653 24904 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:53.627188 24904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Generated new TSK 0
I20260810 07:19:53.627420 24904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab.
Entry for principal oryx/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab.
I20260810 07:19:53.665858 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39011
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:53.753108 24922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:53.753333 24922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:53.753358 24922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:53.754530 24922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:53.754578 24922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:53.754606 24922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:53.756086 24922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:53.756345 24922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:53.756618 24922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:53.762562 24931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.762555 24936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.762555 24930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.762872 24922 server_base.cc:1034] running on GCE node
I20260810 07:19:53.763175 24922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:53.763473 24922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:53.764664 24922 hybrid_clock.cc:648] HybridClock initialized: now 1786346393764634 us; error 66 us; skew 500 ppm
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:53.767663 24922 init.cc:377] Logged in from keytab as oryx/127.14.164.1@KRBTEST.COM (short username oryx)
I20260810 07:19:53.768272 24922 webserver.cc:460] Webserver started at http://127.14.164.1:44023/ using document root <none> and password file <none>
I20260810 07:19:53.768532 24922 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:53.768610 24922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:53.768818 24922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:53.770023 24922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "00791d14d1274bbf82945f94a7ae54c6"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "2302c6584eb892012e81411a3eec288f"
server_key_iv: "9206f711f46b60f2fffb1c291b69e6e6"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.770445 24922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "00791d14d1274bbf82945f94a7ae54c6"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "2302c6584eb892012e81411a3eec288f"
server_key_iv: "9206f711f46b60f2fffb1c291b69e6e6"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.771988 24922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:19:53.772925 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.773125 24922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:53.773238 24922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "00791d14d1274bbf82945f94a7ae54c6"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "2302c6584eb892012e81411a3eec288f"
server_key_iv: "9206f711f46b60f2fffb1c291b69e6e6"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.773329 24922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:53.786376 24922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:53.788707 24922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:53.789006 24922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:53.789287 24922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:53.789640 24922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:53.789700 24922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.789746 24922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:53.789770 24922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.796078 24922 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:37923
I20260810 07:19:53.796139 25134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:37923 every 8 connection(s)
I20260810 07:19:53.796476 24922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:53.802379 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 24922
I20260810 07:19:53.802510 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:53.802757 14992 external_mini_cluster.cc:1428] Setting key 0928ec726492b82b04ab6b3014c602a5
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:53.806967 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.797242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:38285 (local address 127.14.164.62:39011)
0810 07:19:53.797329 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.797333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.797532 (+ 199us) server_negotiation.cc:407] Connection header received
0810 07:19:53.798101 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.798103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.798120 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.798191 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.799463 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.800126 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.801218 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.801421 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.801478 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.803960 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.803961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.803962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.803970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.805493 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.805960 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.805962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.805962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.806027 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.806293 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.806294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.806294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.806415 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.806488 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.806778 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.806867 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:19:53.807185 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.797215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:38285)
0810 07:19:53.797382 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:53.797428 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:53.797973 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:53.798271 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:53.798275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:53.799111 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:53.799328 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.799333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.800219 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.800220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.801134 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.801137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.801499 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.801500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.801603 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.802165 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:53.802174 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:53.803858 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:53.805598 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.805607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.805610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.805866 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.806096 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.806097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.806098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.806216 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.806492 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:53.806500 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:53.806640 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 07:19:53.806754 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 07:19:53.806934 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260810 07:19:53.808007 25136 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39011
I20260810 07:19:53.808136 25136 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:53.808349 25136 heartbeater.cc:511] Master 127.14.164.62:39011 requested a full tablet report, sending...
I20260810 07:19:53.809077 24819 ts_manager.cc:194] Registered new tserver with Master: 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923)
I20260810 07:19:53.810194 24819 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:38285
WARNING: no policy specified for oryx/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab.
Entry for principal oryx/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab.
I20260810 07:19:53.833276 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39011
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:53.923707 25145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:53.923925 25145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:53.923949 25145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:53.925225 25145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:53.925271 25145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:53.925303 25145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:53.927364 25145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:53.927613 25145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:53.927891 25145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:53.933280 25153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.933282 25151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.933286 25159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.933434 25145 server_base.cc:1034] running on GCE node
I20260810 07:19:53.933705 25145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:53.933977 25145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:53.935148 25145 hybrid_clock.cc:648] HybridClock initialized: now 1786346393935137 us; error 46 us; skew 500 ppm
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:53.937767 25145 init.cc:377] Logged in from keytab as oryx/127.14.164.2@KRBTEST.COM (short username oryx)
I20260810 07:19:53.938225 25145 webserver.cc:460] Webserver started at http://127.14.164.2:44815/ using document root <none> and password file <none>
I20260810 07:19:53.938469 25145 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:53.938542 25145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:53.938706 25145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:53.939810 25145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "9d8d3fb6a0164924bfba34125b1747f9"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "17d347ad1cb391516024f9334235913e"
server_key_iv: "ecb4ad5b86e5dc7c2571e9db6a7c2cac"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.940192 25145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "9d8d3fb6a0164924bfba34125b1747f9"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "17d347ad1cb391516024f9334235913e"
server_key_iv: "ecb4ad5b86e5dc7c2571e9db6a7c2cac"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.941663 25145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:53.942449 25170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.942662 25145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:53.942750 25145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "9d8d3fb6a0164924bfba34125b1747f9"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "17d347ad1cb391516024f9334235913e"
server_key_iv: "ecb4ad5b86e5dc7c2571e9db6a7c2cac"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.942817 25145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:53.951668 25145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:53.954022 25145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:53.954133 25145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:53.954334 25145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:53.954586 25145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:53.954614 25145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.954638 25145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:53.954649 25145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.962250 25145 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:42947
I20260810 07:19:53.962293 25358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:42947 every 8 connection(s)
I20260810 07:19:53.962683 25145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:53.969430 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 25145
I20260810 07:19:53.969519 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
Aug 10 07:19:53 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:53.969771 14992 external_mini_cluster.cc:1428] Setting key 3df96d873699bb7b4a0ed319681fbb14
I20260810 07:19:53.973428 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.963871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:38389 (local address 127.14.164.62:39011)
0810 07:19:53.963945 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.963947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.963960 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:53.964569 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.964570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.964584 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.964639 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.965925 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.966463 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.967546 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.967771 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.967853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.970198 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.970199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.970199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.970208 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.971722 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.972292 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.972294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.972294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.972369 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.972690 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.972690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.972691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.972874 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.972935 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.973290 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.973362 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:53.973711 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.963522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:38389)
0810 07:19:53.963714 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:53.963763 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:53.964452 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:53.964740 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:53.964744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:53.965577 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:53.965818 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.965823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.966567 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.966568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.967449 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.967452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.967862 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.967863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.967992 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.968509 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:53.968517 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:53.970092 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:53.971819 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.971827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.971830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.972194 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.972464 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.972465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.972466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.972618 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.972944 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:53.972951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:53.973126 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 07:19:53.973253 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:19:53.973436 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260810 07:19:53.974273 25362 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39011
I20260810 07:19:53.974375 25362 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:53.974577 25362 heartbeater.cc:511] Master 127.14.164.62:39011 requested a full tablet report, sending...
I20260810 07:19:53.974988 24819 ts_manager.cc:194] Registered new tserver with Master: 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947)
I20260810 07:19:53.975466 24819 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:38389
WARNING: no policy specified for oryx/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab.
Entry for principal oryx/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab.
I20260810 07:19:54.000897 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39011
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:54.089596 25371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:54.089861 25371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:54.089912 25371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:54.091158 25371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:54.091233 25371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:54.091279 25371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:54.092917 25371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:54.093196 25371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:54.093452 25371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.098822 25379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.098869 25384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.098884 25371 server_base.cc:1034] running on GCE node
W20260810 07:19:54.098829 25381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.099264 25371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.099530 25371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.100687 25371 hybrid_clock.cc:648] HybridClock initialized: now 1786346394100676 us; error 52 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.103458 25371 init.cc:377] Logged in from keytab as oryx/127.14.164.3@KRBTEST.COM (short username oryx)
I20260810 07:19:54.104014 25371 webserver.cc:460] Webserver started at http://127.14.164.3:39789/ using document root <none> and password file <none>
I20260810 07:19:54.104264 25371 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.104336 25371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.104496 25371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:54.105651 25371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "e6a91f9597b843e398d0a46954830499"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "71061a81a9cb0f096b20022541c3075a"
server_key_iv: "e1b36dd8df36d95a533cf17220cd7711"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.106045 25371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "e6a91f9597b843e398d0a46954830499"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "71061a81a9cb0f096b20022541c3075a"
server_key_iv: "e1b36dd8df36d95a533cf17220cd7711"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.107457 25371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:54.108461 25392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.108661 25371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:54.108847 25371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "e6a91f9597b843e398d0a46954830499"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "71061a81a9cb0f096b20022541c3075a"
server_key_iv: "e1b36dd8df36d95a533cf17220cd7711"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.108944 25371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.123486 25371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.126006 25371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:54.126178 25371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.126471 25371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:54.127058 25371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:54.127213 25371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.127308 25371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:54.127379 25371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.133661 25371 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:45577
I20260810 07:19:54.133706 25582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:45577 every 8 connection(s)
I20260810 07:19:54.134050 25371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:54.137328 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 25371
I20260810 07:19:54.137444 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:54.137683 14992 external_mini_cluster.cc:1428] Setting key 5b2c30ab83e12523410a280f6be92d70
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:54.144191 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.134763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:46003 (local address 127.14.164.62:39011)
0810 07:19:54.134846 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.134849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.135102 (+ 253us) server_negotiation.cc:407] Connection header received
0810 07:19:54.135630 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.135631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.135642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.135696 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.136702 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.137372 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.138546 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.138782 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.138862 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.141199 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.141201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.141201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.141210 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.142722 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.143161 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.143162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.143163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.143227 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.143496 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.143497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.143498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.143622 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.143699 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.144000 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.144094 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:19:54.144399 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.134738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:46003)
0810 07:19:54.134936 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.134975 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.135541 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.135750 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.135754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.136408 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.136611 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.136616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.137491 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.137492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.138446 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.138448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.138884 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.138885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.139010 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.139605 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.139614 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.141094 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.142831 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.142839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.142841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.143079 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.143300 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.143301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.143302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.143424 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.143700 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.143709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.143865 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.143977 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.144147 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":118,"threads_started":1}
I20260810 07:19:54.145006 25583 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39011
I20260810 07:19:54.145114 25583 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:54.145283 25583 heartbeater.cc:511] Master 127.14.164.62:39011 requested a full tablet report, sending...
I20260810 07:19:54.145713 24819 ts_manager.cc:194] Registered new tserver with Master: e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577)
I20260810 07:19:54.146127 24819 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:46003
I20260810 07:19:54.149437 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.164.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.61.keytab.
Entry for principal oryx/127.14.164.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.61.keytab.
I20260810 07:19:54.176777 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:46579
--webserver_interface=127.14.164.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.61.keytab
--principal=oryx/127.14.164.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39011,127.14.164.61:46579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:54.261693 25591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:54.261956 25591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:54.261994 25591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:54.263149 25591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:54.263221 25591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:54.263247 25591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:54.263267 25591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:54.263296 25591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:54.264814 25591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39011,127.14.164.61:46579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.61.keytab
--principal=oryx/127.14.164.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:46579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.164.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
W20260810 07:19:54.264969 25591 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.164.62:39011,127.14.164.61:46579'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 07:19:54.265133 25591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:54.265365 25591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.270455 25600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.270485 25599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.270586 25602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.270779 25591 server_base.cc:1034] running on GCE node
I20260810 07:19:54.271126 25591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.271427 25591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.272574 25591 hybrid_clock.cc:648] HybridClock initialized: now 1786346394272552 us; error 39 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.275307 25591 init.cc:377] Logged in from keytab as oryx/127.14.164.61@KRBTEST.COM (short username oryx)
I20260810 07:19:54.275898 25591 webserver.cc:460] Webserver started at http://127.14.164.61:36247/ using document root <none> and password file <none>
I20260810 07:19:54.276180 25591 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.276226 25591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.276371 25591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:54.277643 25591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/instance:
uuid: "44c5429b0e6843d188c47d116833847f"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "07038a8a6a1401ce6a9a5e6bad25a6b5"
server_key_iv: "e3d3ed8735f1fc6e6ac2f3cf3aa47fb2"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.278108 25591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal/instance:
uuid: "44c5429b0e6843d188c47d116833847f"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "07038a8a6a1401ce6a9a5e6bad25a6b5"
server_key_iv: "e3d3ed8735f1fc6e6ac2f3cf3aa47fb2"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.279541 25591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:54.280403 25612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.280630 25591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:54.280756 25591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
uuid: "44c5429b0e6843d188c47d116833847f"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "07038a8a6a1401ce6a9a5e6bad25a6b5"
server_key_iv: "e3d3ed8735f1fc6e6ac2f3cf3aa47fb2"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.280869 25591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.297585 25591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.300190 25591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:54.300364 25591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.304793 25591 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.61:46579
I20260810 07:19:54.304854 25688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.61:46579 every 8 connection(s)
I20260810 07:19:54.305197 25591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
I20260810 07:19:54.306208 25689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.307727 25689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:54.312639 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 25591
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.61@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:54.318090 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.308432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53422)
0810 07:19:54.308640 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.308782 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.309312 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.309552 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.309556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.309836 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.310050 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.310055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.310827 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.310828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.311762 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.311766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.312169 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.312170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.312282 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.312915 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.312926 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.314472 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.316385 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.316394 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.316396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.316664 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.317010 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.317012 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.317012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.317166 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.317584 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.317590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.317708 (+ 118us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.317801 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.317927 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260810 07:19:54.318118 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.308451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.14.164.62:39011)
0810 07:19:54.308522 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.308524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.308878 (+ 354us) server_negotiation.cc:407] Connection header received
0810 07:19:54.309418 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.309419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.309430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.309494 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.310160 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.310687 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.311867 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.312069 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.312130 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.314558 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.314560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.314560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.314569 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.316242 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.316831 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.316833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.316833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.316904 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.317275 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.317276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.317277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.317501 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.317584 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.317894 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.317989 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:19:54.318804 25689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
I20260810 07:19:54.322852 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.313324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.14.164.61:46579)
0810 07:19:54.313613 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.313617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.313635 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:19:54.313652 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.313654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.313670 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.313752 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.314435 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.315498 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.316235 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.316574 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.316663 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.319667 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.319674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.319675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.319682 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.321518 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.321934 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.321937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.321937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.322016 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.322217 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.322218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.322218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.322400 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.322590 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.322643 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.322714 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":84,"threads_started":1}
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.61@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
I20260810 07:19:54.326484 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.319062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56634)
0810 07:19:54.319114 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.319126 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.319171 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.319481 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.319482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.319602 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.319676 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.319678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.320294 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.320295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.321114 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.321117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.321495 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.321496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.321581 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.322001 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.322006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.323430 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.325472 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.325472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.325473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.325637 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.325857 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.325858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.325858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.325913 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.326158 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.326159 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.326209 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.326291 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.326386 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 07:19:54.326581 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.319135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.14.164.61:46579)
0810 07:19:54.319347 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.319349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.319357 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:54.319379 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.319379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.319391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.319468 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.319756 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.320218 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.321203 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.321413 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.321474 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.323520 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.323521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.323522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.323529 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.325393 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.325707 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.325708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.325708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.325784 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.325969 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.325970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.325970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.326067 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.326120 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.326323 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.326433 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":178,"thread_start_us":124,"threads_started":1}
I20260810 07:19:54.328009 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Bootstrap starting.
I20260810 07:19:54.328661 25689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.328971 25689 log.cc:824] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:54.329758 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: No bootstrap required, opened a new log
I20260810 07:19:54.331188 25689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } }
I20260810 07:19:54.331322 25689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.331346 25689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c5429b0e6843d188c47d116833847f, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.331481 25689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } }
I20260810 07:19:54.331808 25689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:54.331795 25699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } } }
I20260810 07:19:54.331923 25699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:54.333710 25719 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:54.333771 25719 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:54.335891 25591 master_runner.cc:428] Detected that this master 44c5429b0e6843d188c47d116833847f is joining an existing cluster
I20260810 07:19:54.335944 25591 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.164.61:46579
I20260810 07:19:54.345044 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.337444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.14.164.61:46579)
0810 07:19:54.337488 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.337491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.337565 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:19:54.337648 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.337649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.337662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.337737 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.338188 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.338764 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.339976 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.340301 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.340373 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.341686 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.341688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.341688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.341694 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.343631 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.344026 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.344027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.344028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.344116 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.344357 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.344358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.344359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.344556 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.344640 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.344845 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.344931 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:19:54.345387 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.337271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56640)
0810 07:19:54.337484 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.337498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.337575 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.337743 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.337744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.337996 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.338085 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.338088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.338887 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.338888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.339874 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.339877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.340442 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.340443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.340506 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.341173 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.341178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.341593 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.343719 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.343720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.343720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.343934 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.344202 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.344202 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.344203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.344274 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.344625 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.344626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.344673 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.345187 (+ 514us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.345285 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":75,"threads_started":1}
I20260810 07:19:54.345440 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.337349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.14.164.62:39011)
0810 07:19:54.337405 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.337408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.337572 (+ 164us) server_negotiation.cc:407] Connection header received
0810 07:19:54.337699 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.337700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.337715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.337780 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.338238 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.339003 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.340062 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.340295 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.340398 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.342060 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.342062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.342063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.342071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.343869 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.344427 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.344428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.344429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.344492 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.344793 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.344794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.344794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.344940 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.345034 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.345238 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.345340 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:19:54.345852 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.337308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53430)
0810 07:19:54.337506 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.337516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.337600 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.337840 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.337840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.338054 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.338121 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.338123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.339095 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.339095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.339987 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.339989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.340435 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.340437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.340519 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.341530 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.341535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.341901 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.343993 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.343994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.343995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.344288 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.344571 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.344572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.344572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.344664 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.345025 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.345026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.345075 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.345569 (+ 494us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.345724 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.62@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
I20260810 07:19:54.355974 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.348650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.14.164.61:46579)
0810 07:19:54.348712 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.348714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.348771 (+ 57us) server_negotiation.cc:407] Connection header received
0810 07:19:54.348897 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.348898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.348907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.348958 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.349433 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.349902 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.350694 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.350901 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.350975 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.353289 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.353290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.353291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.353298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.354710 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.355110 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.355111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.355111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.355169 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.355386 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.355387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.355388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.355512 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.355587 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.355786 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.355874 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:19:54.356003 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.348468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56644)
0810 07:19:54.348639 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.348704 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.348800 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.348962 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.348965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.349264 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.349346 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.349348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.349988 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.349989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.350595 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.350597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.350976 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.350977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.351059 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.351576 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.351581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.353194 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.354803 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.354811 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.354814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.355032 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.355247 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.355247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.355248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.355314 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.355580 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.355589 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.355645 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.355741 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.355879 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260810 07:19:54.356981 24819 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.164.61:46579: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "44c5429b0e6843d188c47d116833847f"
member_type: NON_VOTER
last_known_addr {
host: "127.14.164.61"
port: 46579
}
attrs {
promote: true
}
}
dest_uuid: "b6e26534bc5a42bf8955212b6e8295c8"
cas_config_opid_index: -1
I20260810 07:19:54.357378 24819 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } }
I20260810 07:19:54.357884 25734 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 44c5429b0e6843d188c47d116833847f (127.14.164.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } }
I20260810 07:19:54.358211 25649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.358958 25734 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.164.61:46579
I20260810 07:19:54.359179 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } }
I20260810 07:19:54.359221 25649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Refusing update from remote peer b6e26534bc5a42bf8955212b6e8295c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 07:19:54.359385 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:54.359401 25591 master.cc:537] Master@127.14.164.61:46579 shutting down...
I20260810 07:19:54.359580 25734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:19:54.360383 24798 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 07:19:54.360893 24798 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 -> Peer 44c5429b0e6843d188c47d116833847f (127.14.164.61:46579): Couldn't send request to peer 44c5429b0e6843d188c47d116833847f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:19:54.362596 25591 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:54.362692 25591 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:54.362756 25591 tablet_replica.cc:330] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: stopping tablet replica
I20260810 07:19:54.375540 25591 master.cc:552] Master@127.14.164.61:46579 shutdown complete.
I20260810 07:19:54.375597 25591 master_runner.cc:305] Clearing existing system tablet
I20260810 07:19:54.375753 25591 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.375811 25591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.376544 25591 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260810 07:19:54.377178 25741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.377322 25591 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.377396 25591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.377424 25591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
uuid: "44c5429b0e6843d188c47d116833847f"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "07038a8a6a1401ce6a9a5e6bad25a6b5"
server_key_iv: "e3d3ed8735f1fc6e6ac2f3cf3aa47fb2"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.378337 25591 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:54.379935 25591 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:19:54.380012 25591 log.cc:1196] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 07:19:54.380218 25591 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Deleting consensus metadata
I20260810 07:19:54.380409 25591 master_runner.cc:315] Copying system tablet from 127.14.164.62:39011
I20260810 07:19:54.381068 25591 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: tablet copy: Beginning tablet copy session from remote peer at address 127.14.164.62:39011
I20260810 07:19:54.388036 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.381579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.14.164.62:39011)
0810 07:19:54.381629 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.381632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.381679 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:19:54.381764 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.381765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.381776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.381829 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.382204 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.382696 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.383790 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.383995 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.384062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.385319 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.385320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.385321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.385327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.386878 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.387285 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.387287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.387287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.387346 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.387554 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.387554 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.387555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.387659 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.387710 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.387883 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.387948 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:19:54.388080 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.381474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53440)
0810 07:19:54.381610 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.381621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.381696 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.381837 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.381838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.382052 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.382134 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.382136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.382777 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.382778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.383681 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.383684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.384083 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.384084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.384157 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.384787 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.384792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.385212 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.386964 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.386965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.386966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.387203 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.387418 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.387419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.387419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.387490 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.387724 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.387725 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.387775 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.387856 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.387967 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260810 07:19:54.388871 24851 tablet_copy_service.cc:140] P b6e26534bc5a42bf8955212b6e8295c8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 44c5429b0e6843d188c47d116833847f ({username='oryx', principal='oryx/127.14.164.61@KRBTEST.COM'} at 127.0.0.1:53440)
I20260810 07:19:54.389029 24851 tablet_copy_service.cc:161] P b6e26534bc5a42bf8955212b6e8295c8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 44c5429b0e6843d188c47d116833847f at {username='oryx', principal='oryx/127.14.164.61@KRBTEST.COM'} at 127.0.0.1:53440: session id = 44c5429b0e6843d188c47d116833847f-00000000000000000000000000000000
I20260810 07:19:54.389999 24851 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:19:54.390772 25591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:19:54.392272 25591 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: tablet copy: Starting download of 0 data blocks...
I20260810 07:19:54.392457 25591 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: tablet copy: Starting download of 1 WAL segments...
I20260810 07:19:54.393793 25591 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:19:54.394824 24851 tablet_copy_service.cc:342] P b6e26534bc5a42bf8955212b6e8295c8: Request end of tablet copy session 44c5429b0e6843d188c47d116833847f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.164.61@KRBTEST.COM'} at 127.0.0.1:53440
I20260810 07:19:54.394887 24851 tablet_copy_service.cc:433] P b6e26534bc5a42bf8955212b6e8295c8: ending tablet copy session 44c5429b0e6843d188c47d116833847f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 44c5429b0e6843d188c47d116833847f
I20260810 07:19:54.397629 25591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.399083 25751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.399109 25754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.399320 25756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.399446 25591 server_base.cc:1034] running on GCE node
I20260810 07:19:54.399565 25591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.399739 25591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.400867 25591 hybrid_clock.cc:648] HybridClock initialized: now 1786346394400855 us; error 26 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.402511 25591 init.cc:377] Logged in from keytab as oryx/127.14.164.61@KRBTEST.COM (short username oryx)
I20260810 07:19:54.402819 25591 webserver.cc:460] Webserver started at http://127.14.164.61:42207/ using document root <none> and password file <none>
I20260810 07:19:54.402987 25591 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.403038 25591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.403705 25591 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:19:54.404282 25767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.404446 25591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:19:54.404506 25591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
uuid: "44c5429b0e6843d188c47d116833847f"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "07038a8a6a1401ce6a9a5e6bad25a6b5"
server_key_iv: "e3d3ed8735f1fc6e6ac2f3cf3aa47fb2"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.404563 25591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.410892 25591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.412670 25591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.416003 25591 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.61:46579
I20260810 07:19:54.416457 25591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
I20260810 07:19:54.416651 25851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.61:46579 every 8 connection(s)
I20260810 07:19:54.417348 25853 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:19:54.417779 25853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Bootstrap starting.
I20260810 07:19:54.420009 25853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:19:54.420243 25853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Bootstrap complete.
I20260810 07:19:54.420455 25853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } }
I20260810 07:19:54.420545 25853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 44c5429b0e6843d188c47d116833847f, State: Initialized, Role: LEARNER
I20260810 07:19:54.420590 25853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } }
I20260810 07:19:54.420833 25853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:54.420804 25856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } }
I20260810 07:19:54.420894 25856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: LEARNER
I20260810 07:19:54.422382 25869 catalog_manager.cc:1259] Loaded cluster ID: ef8aacfb45514870bf4025d483ab3a5f
I20260810 07:19:54.422453 25869 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: loading cluster ID for follower catalog manager: success
I20260810 07:19:54.423525 25869 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: acquiring CA information for follower catalog manager: success
I20260810 07:19:54.423810 25869 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:54.431134 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.424586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.14.164.61:46579)
0810 07:19:54.424702 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.424704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.424712 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:54.424755 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.424755 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.424766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.424847 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.425383 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.426035 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.426748 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.426975 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.427056 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.428308 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.428310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.428310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.428318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.429890 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.430281 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.430282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.430283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.430349 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.430555 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.430556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.430556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.430673 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.430793 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.430894 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.430991 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
WARNING: no policy specified for oryx/127.14.164.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.60.keytab.
Entry for principal oryx/127.14.164.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.60.keytab.
I20260810 07:19:54.469280 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:39157
--webserver_interface=127.14.164.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.60.keytab
--principal=oryx/127.14.164.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:54.568133 25877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:54.568341 25877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:54.568363 25877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:54.569619 25877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:54.569662 25877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:54.569677 25877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:54.569689 25877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:54.569701 25877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:54.571218 25877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.60.keytab
--principal=oryx/127.14.164.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:39157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.164.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:54.571486 25877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:54.571729 25877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.576596 25888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.576613 25886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.576594 25883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.577091 25877 server_base.cc:1034] running on GCE node
I20260810 07:19:54.577363 25877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.577678 25877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.578847 25877 hybrid_clock.cc:648] HybridClock initialized: now 1786346394578838 us; error 61 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.581478 25877 init.cc:377] Logged in from keytab as oryx/127.14.164.60@KRBTEST.COM (short username oryx)
I20260810 07:19:54.581987 25877 webserver.cc:460] Webserver started at http://127.14.164.60:44681/ using document root <none> and password file <none>
I20260810 07:19:54.582250 25877 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.582321 25877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.582479 25877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:54.583546 25877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/instance:
uuid: "ab7dc73389884a5aa28a857c270f4f12"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "22565d0bdaabba489ade5f9108ccdfac"
server_key_iv: "22559f5b2b82eb7c315e6b36ba156937"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.583918 25877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal/instance:
uuid: "ab7dc73389884a5aa28a857c270f4f12"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "22565d0bdaabba489ade5f9108ccdfac"
server_key_iv: "22559f5b2b82eb7c315e6b36ba156937"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.585422 25877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:54.586179 25904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.586438 25877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:54.586549 25877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
uuid: "ab7dc73389884a5aa28a857c270f4f12"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "22565d0bdaabba489ade5f9108ccdfac"
server_key_iv: "22559f5b2b82eb7c315e6b36ba156937"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.586634 25877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.595098 25877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.597612 25877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:54.597788 25877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.602830 25877 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.60:39157
I20260810 07:19:54.602875 25984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.60:39157 every 8 connection(s)
I20260810 07:19:54.603309 25877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
I20260810 07:19:54.604388 25986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.606317 25986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:54.606839 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 25877
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.60@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.164.60@KRBTEST.COM
I20260810 07:19:54.616709 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.607187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53450 (local address 127.14.164.62:39011)
0810 07:19:54.607279 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.607282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.607514 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:19:54.608159 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.608160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.608173 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.608231 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.608836 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.609338 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.610445 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.610739 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.610790 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.613380 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.613382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.613382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.613390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.615100 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.615563 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.615564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.615564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.615626 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.615935 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.615936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.615937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.616094 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.616179 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.616535 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.616603 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:19:54.617249 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.607104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53450)
0810 07:19:54.607367 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.607421 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.608028 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.608268 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.608270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.608524 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.608680 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.608684 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.609437 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.609438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.610365 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.610368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.610832 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.610833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.610960 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.611603 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.611616 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.613265 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.615187 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.615192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.615195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.615470 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.615702 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.615702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.615703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.615845 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.616190 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.616197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.616364 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.616503 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.616675 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
I20260810 07:19:54.617249 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.607473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.14.164.60:39157)
0810 07:19:54.607605 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.607609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.607641 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:54.608032 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.608041 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.608068 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.608160 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.608884 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.609898 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.610676 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.611074 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.611150 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.613923 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.613931 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.613932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.613939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.615791 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.616205 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.616208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.616208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.616267 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.616461 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.616461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.616462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.616617 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.616784 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.616984 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.617061 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260810 07:19:54.617854 25986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.60@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
I20260810 07:19:54.624523 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.618117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56682)
0810 07:19:54.618171 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.618183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.618225 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.618402 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.618402 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.618512 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.618541 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.618543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.619193 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.619193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.619691 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.619692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.620042 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.620042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.620085 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.620504 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.620509 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.621982 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.623592 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.623593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.623593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.623753 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.623959 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.623960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.623960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.624034 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.624282 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.624284 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.624324 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.624388 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.624461 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 07:19:54.624531 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.618229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.14.164.61:46579)
0810 07:19:54.618308 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.618310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.618317 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:19:54.618342 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.618343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.618354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.618399 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.618623 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.619121 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.619763 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.619974 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.620032 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.622057 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.622058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.622058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.622065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.623516 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.623820 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.623821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.623821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.623871 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.624098 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.624099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.624099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.624199 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.624250 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.624406 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.624470 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:19:54.625018 25986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.60@KRBTEST.COM for oryx/127.14.164.60@KRBTEST.COM
I20260810 07:19:54.631306 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.625292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39438)
0810 07:19:54.625348 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.625360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.625404 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.625509 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.625510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.625620 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.625649 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.625650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.626398 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.626399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.626846 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.626847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.627219 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.627220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.627259 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.627644 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.627648 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.628814 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.630512 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.630513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.630513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.630670 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.630852 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.630852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.630852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.630902 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.631080 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.631080 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.631112 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.631161 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.631234 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:19:54.631359 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.625355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.14.164.60:39157)
0810 07:19:54.625409 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.625411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.625430 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:19:54.625446 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.625447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.625455 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.625494 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.625756 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.626312 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.626925 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.627149 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.627212 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.629002 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.629003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.629004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.629009 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.630438 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.630734 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.630734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.630734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.630794 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.630941 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.630942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.630942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.631031 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.631074 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.631175 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.631278 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":37}
I20260810 07:19:54.632877 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Bootstrap starting.
I20260810 07:19:54.633821 25986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.634132 25986 log.cc:824] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:54.635047 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: No bootstrap required, opened a new log
I20260810 07:19:54.636547 25986 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } }
I20260810 07:19:54.636698 25986 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.636772 25986 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7dc73389884a5aa28a857c270f4f12, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.636940 25986 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } }
I20260810 07:19:54.637250 25986 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:54.637249 26000 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } } }
I20260810 07:19:54.637389 26000 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:54.639269 26013 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:54.639333 26013 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:54.641891 25877 master_runner.cc:428] Detected that this master ab7dc73389884a5aa28a857c270f4f12 is joining an existing cluster
I20260810 07:19:54.641945 25877 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.164.60:39157
I20260810 07:19:54.650493 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.643444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39450)
0810 07:19:54.643640 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.643649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.643715 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.643879 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.643880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.644076 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.644133 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.644134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.645052 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.645053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.645951 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.645954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.646478 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.646604 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.647150 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.647155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.647544 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.649366 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.649367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.649368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.649558 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.649836 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.649837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.649837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.649894 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.650200 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.650201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.650252 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.650314 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.650415 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":26,"thread_start_us":85,"threads_started":1}
I20260810 07:19:54.650614 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.643374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.14.164.60:39157)
0810 07:19:54.643410 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.643411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.643694 (+ 283us) server_negotiation.cc:407] Connection header received
0810 07:19:54.643788 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.643789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.643798 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.643826 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.644204 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.644938 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646043 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.646332 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646371 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.647642 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.647644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.647644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.647652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.649251 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.649658 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.649660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.649660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.649761 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.649972 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.649973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.649974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.650129 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.650202 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.650366 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.650493 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":19}
I20260810 07:19:54.665390 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.643034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.14.164.62:39011)
0810 07:19:54.643113 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.643115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.643341 (+ 226us) server_negotiation.cc:407] Connection header received
0810 07:19:54.643447 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.643447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.643459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.643490 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.644031 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.644864 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646246 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.646564 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646607 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.662141 (+ 15534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.662143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.662144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.662152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.664001 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664437 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.664438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.664439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.664504 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664778 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.664779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.664779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.664957 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.665033 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.665197 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.665287 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:19:54.665417 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.643113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53454)
0810 07:19:54.643270 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.643282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.643370 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.643557 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.643558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.643855 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.643934 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.643936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.645125 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.645126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.646107 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.646110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.646710 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.646792 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.661377 (+ 14585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.661384 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.661975 (+ 591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.664101 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.664103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.664341 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.664592 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.664593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.664664 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.665036 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.665037 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.665087 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.665170 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.665288 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260810 07:19:54.665717 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.643051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56696)
0810 07:19:54.643270 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.643282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.643360 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.643565 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.643566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.643926 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.643999 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.644001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.645003 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.645004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.646023 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.646025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.646525 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.646599 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.661441 (+ 14842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.661446 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.661930 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.664270 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.664271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.664533 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.664792 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.664793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.664851 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.665215 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.665216 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.665261 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.665324 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.665449 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":137,"thread_start_us":84,"threads_started":1}
I20260810 07:19:54.666124 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.643159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56696 (local address 127.14.164.61:46579)
0810 07:19:54.643206 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.643208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.643332 (+ 124us) server_negotiation.cc:407] Connection header received
0810 07:19:54.643442 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.643443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.643455 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.643488 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.644087 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.644906 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646105 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.646418 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.646477 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.662059 (+ 15582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.662061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.662062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.662078 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.664186 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664613 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.664614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.664614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.664676 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.664952 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.664954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.664954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.665111 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.665184 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.665925 (+ 741us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.666029 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:19:54.668612 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 24922
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.62@KRBTEST.COM for oryx/127.14.164.60@KRBTEST.COM
I20260810 07:19:54.677838 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:37923
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:39011
--webserver_port=44023
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:54.679203 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.667679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.14.164.60:39157)
0810 07:19:54.667723 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.667725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.667734 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:54.667811 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.667812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.667822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.667877 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.668191 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.668886 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.669649 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.669857 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.669950 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.671840 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.671842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.671842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.671850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.677782 (+ 5932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.678226 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.678228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.678228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.678286 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.678556 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.678557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.678558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.678756 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.678849 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.679018 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.679106 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:19:54.679206 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.667637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39452)
0810 07:19:54.667687 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.667698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.667739 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.667881 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.667882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.668077 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.668128 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.668131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.668972 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.668973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.669564 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.669565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.669951 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.669952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.670037 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.670477 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.670483 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.671745 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.677897 (+ 6152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.677899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.677900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.678144 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.678378 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.678379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.678379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.678449 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.678842 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.678843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.678902 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.678992 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.679106 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 07:19:54.680179 24819 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.164.60:39157: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12"
member_type: NON_VOTER
last_known_addr {
host: "127.14.164.60"
port: 39157
}
attrs {
promote: true
}
}
dest_uuid: "b6e26534bc5a42bf8955212b6e8295c8"
cas_config_opid_index: 5
I20260810 07:19:54.680454 24819 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } }
I20260810 07:19:54.680955 25731 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.682096 25731 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.164.60:39157
I20260810 07:19:54.682170 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.682241 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:54.682497 25877 master.cc:537] Master@127.14.164.60:39157 shutting down...
I20260810 07:19:54.682659 25945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.683668 25945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Refusing update from remote peer b6e26534bc5a42bf8955212b6e8295c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:19:54.684001 25734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:19:54.684929 24801 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 -> Peer ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60:39157): Couldn't send request to peer ab7dc73389884a5aa28a857c270f4f12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:19:54.686568 25877 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:54.686667 25877 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:54.686712 25877 tablet_replica.cc:330] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: stopping tablet replica
I20260810 07:19:54.689455 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.680915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56708)
0810 07:19:54.684814 (+ 3899us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.684828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.684903 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.685274 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.685275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.685277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:54.685327 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.685329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.686342 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.686343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.688118 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.688122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.689139 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.689140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.689220 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.689247 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.689306 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.689357 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3877}
I20260810 07:19:54.689746 25877 master.cc:552] Master@127.14.164.60:39157 shutdown complete.
I20260810 07:19:54.689795 25877 master_runner.cc:305] Clearing existing system tablet
I20260810 07:19:54.689931 25877 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.689982 25877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.690231 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.681002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56708 (local address 127.14.164.61:46579)
0810 07:19:54.684997 (+ 3995us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.685000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.685009 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:54.685036 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.685038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.685068 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.685131 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:54.685416 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.686236 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.688232 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.688939 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.689014 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.689100 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.690087 (+ 987us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.690140 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3963}
I20260810 07:19:54.690645 25808 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 LEARNER]: Refusing update from remote peer b6e26534bc5a42bf8955212b6e8295c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:19:54.690860 25877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:19:54.690994 26027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 07:19:54.691335 26030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.691579 25877 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.691628 25877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.691651 25877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
uuid: "ab7dc73389884a5aa28a857c270f4f12"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "22565d0bdaabba489ade5f9108ccdfac"
server_key_iv: "22559f5b2b82eb7c315e6b36ba156937"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.691658 25808 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.692416 25877 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:54.693295 25856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e26534bc5a42bf8955212b6e8295c8. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.693394 25856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: LEARNER
I20260810 07:19:54.693435 25856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: NON_VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: true } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.693473 25856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: LEARNER
I20260810 07:19:54.693841 25877 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:19:54.693914 25877 log.cc:1196] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 07:19:54.694110 25877 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Deleting consensus metadata
I20260810 07:19:54.694226 25877 master_runner.cc:315] Copying system tablet from 127.14.164.62:39011
I20260810 07:19:54.694283 25734 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: attempting to promote NON_VOTER 44c5429b0e6843d188c47d116833847f to VOTER
I20260810 07:19:54.694478 25734 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } }
I20260810 07:19:54.694865 25877 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: tablet copy: Beginning tablet copy session from remote peer at address 127.14.164.62:39011
I20260810 07:19:54.695082 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.694874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39458)
0810 07:19:54.695012 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.695053 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:39157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 07:19:54.695401 25808 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 LEARNER]: Refusing update from remote peer b6e26534bc5a42bf8955212b6e8295c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 07:19:54.695628 25731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 07:19:54.696049 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.695973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39468)
0810 07:19:54.696004 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.696025 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:39157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260810 07:19:54.696338 24801 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 -> Peer ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60:39157): Couldn't send request to peer ab7dc73389884a5aa28a857c270f4f12. Status: Network error: Client connection negotiation failed: client connection to 127.14.164.60:39157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:19:54.696743 25731 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 44c5429b0e6843d188c47d116833847f (127.14.164.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.697625 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.697696 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:54.698055 25734 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.698123 25734 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:54.698423 25808 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 44c5429b0e6843d188c47d116833847f (127.14.164.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.700026 26032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.700174 26032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:54.703205 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.695220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.14.164.62:39011)
0810 07:19:54.695260 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.695262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.695631 (+ 369us) server_negotiation.cc:407] Connection header received
0810 07:19:54.695740 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.695741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.695752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.695783 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.696223 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.697188 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.698301 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.698605 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.698644 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.700013 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.700014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.700015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.700022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.701908 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.702331 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.702332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.702332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.702399 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.702631 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.702632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.702633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.702780 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.702853 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.703031 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.703115 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:54.703223 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.695302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53456)
0810 07:19:54.695567 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.695577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.695657 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.695845 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.695846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.696067 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.696131 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.696133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.697293 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.697294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.698194 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.698197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.698738 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.698740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.698804 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.699405 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.699410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.699855 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.702007 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.702008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.702008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.702247 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.702484 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.702485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.702486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.702561 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.702860 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.702861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.702913 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.702989 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.703102 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":38,"threads_started":1}
I20260810 07:19:54.703485 24851 tablet_copy_service.cc:140] P b6e26534bc5a42bf8955212b6e8295c8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ab7dc73389884a5aa28a857c270f4f12 ({username='oryx', principal='oryx/127.14.164.60@KRBTEST.COM'} at 127.0.0.1:53456)
I20260810 07:19:54.703541 24851 tablet_copy_service.cc:161] P b6e26534bc5a42bf8955212b6e8295c8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ab7dc73389884a5aa28a857c270f4f12 at {username='oryx', principal='oryx/127.14.164.60@KRBTEST.COM'} at 127.0.0.1:53456: session id = ab7dc73389884a5aa28a857c270f4f12-00000000000000000000000000000000
I20260810 07:19:54.704113 24851 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:19:54.704692 25877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:19:54.706072 25877 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: tablet copy: Starting download of 0 data blocks...
I20260810 07:19:54.706240 25877 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: tablet copy: Starting download of 1 WAL segments...
I20260810 07:19:54.707116 25877 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:19:54.708122 24851 tablet_copy_service.cc:342] P b6e26534bc5a42bf8955212b6e8295c8: Request end of tablet copy session ab7dc73389884a5aa28a857c270f4f12-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.164.60@KRBTEST.COM'} at 127.0.0.1:53456
I20260810 07:19:54.708176 24851 tablet_copy_service.cc:433] P b6e26534bc5a42bf8955212b6e8295c8: ending tablet copy session ab7dc73389884a5aa28a857c270f4f12-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ab7dc73389884a5aa28a857c270f4f12
I20260810 07:19:54.711225 25877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.712559 26047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.713680 26048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.713948 26054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.714076 25877 server_base.cc:1034] running on GCE node
I20260810 07:19:54.714248 25877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.714397 25877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.715518 25877 hybrid_clock.cc:648] HybridClock initialized: now 1786346394715508 us; error 25 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.717222 25877 init.cc:377] Logged in from keytab as oryx/127.14.164.60@KRBTEST.COM (short username oryx)
I20260810 07:19:54.717509 25877 webserver.cc:460] Webserver started at http://127.14.164.60:43719/ using document root <none> and password file <none>
I20260810 07:19:54.717621 25877 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.717648 25877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.718423 25877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:54.719040 26068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.719154 25877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.719192 25877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
uuid: "ab7dc73389884a5aa28a857c270f4f12"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "22565d0bdaabba489ade5f9108ccdfac"
server_key_iv: "22559f5b2b82eb7c315e6b36ba156937"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.719254 25877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.738749 25877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.740763 25877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.744053 25877 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.60:39157
I20260810 07:19:54.744092 26148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.60:39157 every 8 connection(s)
I20260810 07:19:54.744434 25877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
I20260810 07:19:54.745698 26149 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:19:54.746318 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Bootstrap starting.
I20260810 07:19:54.749684 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:19:54.749929 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: Bootstrap complete.
I20260810 07:19:54.750159 26149 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } }
I20260810 07:19:54.750264 26149 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ab7dc73389884a5aa28a857c270f4f12, State: Initialized, Role: LEARNER
I20260810 07:19:54.750308 26149 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } }
I20260810 07:19:54.750542 26149 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:54.752125 26174 catalog_manager.cc:1259] Loaded cluster ID: ef8aacfb45514870bf4025d483ab3a5f
I20260810 07:19:54.752177 26174 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: loading cluster ID for follower catalog manager: success
I20260810 07:19:54.753931 26174 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: acquiring CA information for follower catalog manager: success
I20260810 07:19:54.754236 26174 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:54.756889 26153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:54.757027 26153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: This master's current role is: LEARNER
W20260810 07:19:54.781237 26024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:54.781507 26024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:54.781551 26024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:54.783048 26024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:54.783136 26024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:54.783188 26024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:19:54.785037 26024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:37923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=44023
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:54.785343 26024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:54.785629 26024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.791546 26184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.791716 26189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.791993 26024 server_base.cc:1034] running on GCE node
W20260810 07:19:54.791527 26183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.792305 26024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.792609 26024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.793798 26024 hybrid_clock.cc:648] HybridClock initialized: now 1786346394793784 us; error 35 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.796605 26024 init.cc:377] Logged in from keytab as oryx/127.14.164.1@KRBTEST.COM (short username oryx)
I20260810 07:19:54.797153 26024 webserver.cc:460] Webserver started at http://127.14.164.1:44023/ using document root <none> and password file <none>
I20260810 07:19:54.797429 26024 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.797497 26024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.798981 26024 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:19:54.799692 26200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.799896 26024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:54.799991 26024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "00791d14d1274bbf82945f94a7ae54c6"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "2302c6584eb892012e81411a3eec288f"
server_key_iv: "9206f711f46b60f2fffb1c291b69e6e6"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.800531 26024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.826628 26024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.829221 26024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:54.829411 26024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.829690 26024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:54.830031 26024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:54.830088 26024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.830129 26024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:54.830159 26024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.837021 26024 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:37923
I20260810 07:19:54.837139 26372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:37923 every 8 connection(s)
I20260810 07:19:54.837481 26024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:54.844789 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 26024
I20260810 07:19:54.844981 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 25145
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.60@KRBTEST.COM
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:54.851065 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.838321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:51733 (local address 127.14.164.61:46579)
0810 07:19:54.838514 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.838516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.838881 (+ 365us) server_negotiation.cc:407] Connection header received
0810 07:19:54.839827 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.839828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.839844 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.839878 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.841654 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.842371 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843264 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.843557 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843636 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.846977 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.846979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.846979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.846989 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.849435 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.850018 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.850020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.850020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.850076 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.850338 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.850338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.850339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.850480 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.850518 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.850846 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.850930 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260810 07:19:54.851423 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.838323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.14.164.1:43107)
0810 07:19:54.838523 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.838567 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.839676 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.839967 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.839971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.840407 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.840618 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.840623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.841869 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.841870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.842808 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.842811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.843326 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.843485 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.844173 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.844178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.846721 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.849399 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.849409 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.849412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.849942 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.850198 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.850199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.850199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.850300 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.850681 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.850682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.850735 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.850833 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.851076 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":30,"thread_start_us":116,"threads_started":1}
I20260810 07:19:54.851432 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.838323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.14.164.1:51733)
0810 07:19:54.838810 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.838821 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.839707 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.839976 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.839977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.840409 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.840746 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.840749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.842462 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.842463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.843161 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.843163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.843651 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.843722 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.844356 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.844361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.846875 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.849521 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.849522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.849523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.849939 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.850137 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.850138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.850138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.850276 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.850556 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.850561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.850716 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.851151 (+ 435us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.851266 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":29,"thread_start_us":115,"threads_started":1}
I20260810 07:19:54.851944 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.838409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:43107 (local address 127.14.164.60:39157)
0810 07:19:54.838569 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.838571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.838662 (+ 91us) server_negotiation.cc:407] Connection header received
0810 07:19:54.839794 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.839795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.839810 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.839883 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.840742 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.841769 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.842912 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.843244 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843328 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.846855 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.846856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.846856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.846866 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.849278 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.850040 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.850042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.850043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.850120 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.850397 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.850398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.850399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.850579 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.850651 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.851684 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.851795 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":55,"threads_started":1}
I20260810 07:19:54.852389 26375 heartbeater.cc:347] Connected to a master server at 127.14.164.60:39157
I20260810 07:19:54.852496 26375 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:54.852684 26375 heartbeater.cc:511] Master 127.14.164.60:39157 requested a full tablet report, sending...
I20260810 07:19:54.853930 26081 ts_manager.cc:194] Registered new tserver with Master: 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923)
I20260810 07:19:54.855858 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.838753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:58087 (local address 127.14.164.62:39011)
0810 07:19:54.838809 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.838812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.838932 (+ 120us) server_negotiation.cc:407] Connection header received
0810 07:19:54.839841 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.839843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.839855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.839885 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.840803 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.841551 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.842976 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.843262 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843321 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.847331 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.847333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.847333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.847342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.854575 (+ 7233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.855083 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.855085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.855085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.855145 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.855391 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.855391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.855392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.855497 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.855548 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.855733 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.855794 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:19:54.856045 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.838655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:58087)
0810 07:19:54.838871 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.838881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.839743 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.839973 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.839974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.840412 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.840620 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.840636 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.841772 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.841773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.842841 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.842843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.843386 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.843387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.843498 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.844107 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.844115 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.847195 (+ 3080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.854677 (+ 7482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.854679 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.854680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.855000 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.855224 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.855226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.855226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.855320 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.855565 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.855567 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.855624 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.855721 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.855868 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":41,"thread_start_us":61,"threads_started":1}
I20260810 07:19:54.858767 26377 heartbeater.cc:347] Connected to a master server at 127.14.164.61:46579
I20260810 07:19:54.858783 26376 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39011
I20260810 07:19:54.858848 26377 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:54.858865 26376 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:54.858927 26376 heartbeater.cc:511] Master 127.14.164.62:39011 requested a full tablet report, sending...
I20260810 07:19:54.858935 26377 heartbeater.cc:511] Master 127.14.164.61:46579 requested a full tablet report, sending...
I20260810 07:19:54.859232 24819 ts_manager.cc:194] Re-registered known tserver with Master: 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923)
I20260810 07:19:54.859370 25795 ts_manager.cc:194] Registered new tserver with Master: 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923)
I20260810 07:19:54.859660 24819 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:58087
I20260810 07:19:54.863518 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:42947
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:39011
--webserver_port=44815
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:54.950502 26386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:54.950752 26386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:54.950778 26386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:54.952064 26386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:54.952112 26386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:54.952140 26386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:19:54.953791 26386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:42947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=44815
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:54.954073 26386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:54.954334 26386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.959859 26397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.959903 26386 server_base.cc:1034] running on GCE node
W20260810 07:19:54.959956 26399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.959872 26395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.960362 26386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.960615 26386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.961757 26386 hybrid_clock.cc:648] HybridClock initialized: now 1786346394961753 us; error 39 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.964486 26386 init.cc:377] Logged in from keytab as oryx/127.14.164.2@KRBTEST.COM (short username oryx)
I20260810 07:19:54.965075 26386 webserver.cc:460] Webserver started at http://127.14.164.2:44815/ using document root <none> and password file <none>
I20260810 07:19:54.965328 26386 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.965415 26386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.966866 26386 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:54.967538 26410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.967780 26386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:54.967886 26386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "9d8d3fb6a0164924bfba34125b1747f9"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-x5fp"
server_key: "17d347ad1cb391516024f9334235913e"
server_key_iv: "ecb4ad5b86e5dc7c2571e9db6a7c2cac"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.968256 26386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.981845 26386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.984115 26386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:54.984237 26386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.984537 26386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:54.984929 26386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:54.984987 26386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.985033 26386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:54.985060 26386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.991328 26386 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:42947
I20260810 07:19:54.991374 26584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:42947 every 8 connection(s)
I20260810 07:19:54.991776 26386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.60@KRBTEST.COM
I20260810 07:19:54.999555 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 26386
I20260810 07:19:54.999768 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 25371
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
Aug 10 07:19:54 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:19:55.005299 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.992827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:53511 (local address 127.14.164.60:39157)
0810 07:19:54.992952 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.992954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.993242 (+ 288us) server_negotiation.cc:407] Connection header received
0810 07:19:54.993688 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.993689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.993705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.993741 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.994727 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.995497 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.996522 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.996814 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.996880 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.001736 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.001738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.001739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.001747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.003358 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.003984 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.003987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.003988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.004056 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.004381 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.004382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.004383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.004553 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.004601 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.005069 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.005180 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 07:19:55.005879 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.992646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.14.164.2:53511)
0810 07:19:54.993174 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.993186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.993573 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.993882 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.993883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.994394 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.994592 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.994596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.995588 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.995589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.996424 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.996426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.996887 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.996887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.997034 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.998023 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.998034 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:55.001617 (+ 3583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:55.003483 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.003492 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.003495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.003865 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.004138 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.004139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.004139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.004294 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.004654 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:55.004662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:55.004872 (+ 210us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.005400 (+ 528us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.005607 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":44,"thread_start_us":148,"threads_started":1}
I20260810 07:19:55.007514 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.993023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:34859)
0810 07:19:54.993295 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.993303 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.993578 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.993822 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.993823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.994391 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.994614 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.994616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.995407 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.995407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.996671 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.996673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.997244 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.997245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.997308 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.997809 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.997814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:55.003165 (+ 5351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:55.006247 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.006248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.006248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.006523 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.006746 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.006747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.006747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.006815 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.007091 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:55.007092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:55.007136 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.007262 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.007380 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":83,"thread_start_us":72,"threads_started":1}
I20260810 07:19:55.007586 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.993106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:34859 (local address 127.14.164.62:39011)
0810 07:19:54.993149 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.993152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.993363 (+ 211us) server_negotiation.cc:407] Connection header received
0810 07:19:54.993689 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.993690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.993704 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.993741 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.994722 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.995293 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.996831 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.997154 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.997246 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.003269 (+ 6023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.003271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.003271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.003281 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.005755 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.006605 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.006607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.006607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.006666 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.006897 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.006898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.006898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.007016 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.007087 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.007312 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.007415 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:19:55.008019 26588 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39011
I20260810 07:19:55.008144 26588 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:55.008246 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.992897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:59211 (local address 127.14.164.61:46579)
0810 07:19:54.992993 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.992996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.993008 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:54.993660 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.993662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.993677 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.993739 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.994666 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.995479 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.996786 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.997036 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.997113 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.003678 (+ 6565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.003680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.003680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.003689 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.006123 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.006941 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.006942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.006943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.007009 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.007234 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.007236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.007237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.007386 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.007470 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.008063 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.008151 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:19:55.008392 26588 heartbeater.cc:511] Master 127.14.164.62:39011 requested a full tablet report, sending...
I20260810 07:19:55.008555 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.992645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.14.164.2:59211)
0810 07:19:54.992869 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.992914 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.993551 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.993812 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.993817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.994338 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.994563 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.994568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.995574 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.995575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.996668 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.996671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.997113 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.997114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.997187 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.997743 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.997755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:55.003016 (+ 5261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:55.006615 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.006616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.006616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.006849 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.007080 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.007080 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.007081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.007146 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.007505 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:55.007506 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:55.007548 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.008016 (+ 468us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.008174 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":38,"thread_start_us":136,"threads_started":1}
I20260810 07:19:55.009017 24819 ts_manager.cc:194] Re-registered known tserver with Master: 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947)
I20260810 07:19:55.009030 26585 heartbeater.cc:347] Connected to a master server at 127.14.164.60:39157
I20260810 07:19:55.009083 26585 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:55.009143 26585 heartbeater.cc:511] Master 127.14.164.60:39157 requested a full tablet report, sending...
I20260810 07:19:55.009248 26590 heartbeater.cc:347] Connected to a master server at 127.14.164.61:46579
I20260810 07:19:55.009274 26590 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:55.009318 26590 heartbeater.cc:511] Master 127.14.164.61:46579 requested a full tablet report, sending...
I20260810 07:19:55.009496 26081 ts_manager.cc:194] Registered new tserver with Master: 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947)
I20260810 07:19:55.009527 24819 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:34859
I20260810 07:19:55.009596 25795 ts_manager.cc:194] Registered new tserver with Master: 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947)
I20260810 07:19:55.011999 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:45577
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:39011
--webserver_port=39789
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:55.100117 26597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:55.100417 26597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:55.100462 26597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:55.101728 26597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:55.101807 26597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:55.101856 26597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:19:55.103473 26597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:45577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=39789
--tserver_master_addrs=127.14.164.62:39011,127.14.164.61:46579,127.14.164.60:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:55.103775 26597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:55.104100 26597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:55.107640 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.104805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.14.164.60:39157)
0810 07:19:55.104889 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.104891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.104903 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:55.104936 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.104936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.104961 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.105076 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:55.105241 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.105825 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.106821 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.107275 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.107367 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.107450 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.107523 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.107562 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:19:55.107640 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.104643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39474)
0810 07:19:55.104751 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.104769 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.104844 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.105082 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.105083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.105085 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:55.105144 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.105147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.105904 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.105905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.106720 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.106723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.107367 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.107368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.107459 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.107463 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.107517 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.107567 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:19:55.107959 26106 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 07:19:55.108129 26153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e26534bc5a42bf8955212b6e8295c8. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:55.108251 26153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: This master's current role is: LEARNER
W20260810 07:19:55.110231 26611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:55.110250 26607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:55.110339 26597 server_base.cc:1034] running on GCE node
W20260810 07:19:55.110231 26608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:55.110737 26597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:55.111004 26597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:55.112182 26597 hybrid_clock.cc:648] HybridClock initialized: now 1786346395112168 us; error 29 us; skew 500 ppm
Aug 10 07:19:55 dist-test-slave-x5fp krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346395, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:55.114867 26597 init.cc:377] Logged in from keytab as oryx/127.14.164.3@KRBTEST.COM (short username oryx)
I20260810 07:19:55.115401 26597 webserver.cc:460] Webserver started at http://127.14.164.3:39789/ using document root <none> and password file <none>
I20260810 07:19:55.115674 26597 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:55.115764 26597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:55.117367 26597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:55.118075 26622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:55.118243 26597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:55.118346 26597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "e6a91f9597b843e398d0a46954830499"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-x5fp"
server_key: "71061a81a9cb0f096b20022541c3075a"
server_key_iv: "e1b36dd8df36d95a533cf17220cd7711"
server_key_version: "encryptionkey@0"
I20260810 07:19:55.118716 26597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:55.131011 26597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:55.133580 26597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:55.133704 26597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:55.133949 26597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:55.134318 26597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:55.134364 26597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:55.134389 26597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:55.134428 26597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:55.141395 26597 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:45577
I20260810 07:19:55.141466 26795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:45577 every 8 connection(s)
I20260810 07:19:55.141871 26597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:55.148212 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 26597
Aug 10 07:19:55 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346395, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.61@KRBTEST.COM
Aug 10 07:19:55 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346395, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
Aug 10 07:19:55 dist-test-slave-x5fp krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346395, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.60@KRBTEST.COM
I20260810 07:19:55.154448 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.143023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:45359 (local address 127.14.164.61:46579)
0810 07:19:55.143073 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.143076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.143086 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:55.143844 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.143846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.143860 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.143896 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.144806 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.145357 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.146795 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.147044 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.147098 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.150241 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.150243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.150244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.150254 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.152649 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.153226 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.153228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.153228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.153292 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.153617 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.153618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.153619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.153763 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.153817 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.154244 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.154334 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:19:55.155241 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.143047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:36485)
0810 07:19:55.143256 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.143332 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.143695 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.144076 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.144080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.144670 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:55.144762 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.144765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.145670 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.145670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.146698 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.146700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.147514 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.147515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.147615 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.148111 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:55.148116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:55.150811 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:55.153541 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.153541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.153542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.153748 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.153961 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.153962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.153962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.154016 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.154594 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:55.154596 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:55.154653 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.154725 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.154962 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":31,"thread_start_us":77,"threads_started":1}
I20260810 07:19:55.155241 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.142708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.14.164.3:45359)
0810 07:19:55.142932 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.142980 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.143693 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.144076 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.144080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.144467 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:55.144644 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.144651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.145470 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.145470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.146698 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.146700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.147131 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.147132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.147219 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.147808 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:55.147813 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:55.150014 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:55.152765 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.152774 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.152777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.153124 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.153371 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.153372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.153373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.153534 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.153852 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:55.153860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:55.154062 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.154704 (+ 642us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.155015 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":161,"thread_start_us":97,"threads_started":1}
I20260810 07:19:55.155647 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.143242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:36485 (local address 127.14.164.62:39011)
0810 07:19:55.143298 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.143301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.143383 (+ 82us) server_negotiation.cc:407] Connection header received
0810 07:19:55.143835 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.143837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.143852 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.143892 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.144831 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.145578 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.147149 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.147427 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.147494 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.150945 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.150947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.150947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.150955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.153458 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.153832 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.153833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.153833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.153894 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.154082 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.154083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.154083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.154217 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.154277 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.155425 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.155527 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:19:55.156229 26797 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39011
I20260810 07:19:55.156261 26798 heartbeater.cc:347] Connected to a master server at 127.14.164.61:46579
I20260810 07:19:55.156381 26797 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:55.156401 26798 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:55.156589 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.142698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:52991 (local address 127.14.164.60:39157)
0810 07:19:55.142766 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.142769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.143069 (+ 300us) server_negotiation.cc:407] Connection header received
0810 07:19:55.143860 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.143862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.143882 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.143910 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.144764 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.145574 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.146549 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.146787 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.146855 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.151523 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.151524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.151524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.151534 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.154402 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.155533 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.155535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.155535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.155610 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.155851 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.155852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.155853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.156016 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.156093 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.156394 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.156502 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:19:55.156605 26798 heartbeater.cc:511] Master 127.14.164.61:46579 requested a full tablet report, sending...
I20260810 07:19:55.156610 26797 heartbeater.cc:511] Master 127.14.164.62:39011 requested a full tablet report, sending...
I20260810 07:19:55.156785 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.142709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.14.164.3:52991)
0810 07:19:55.142932 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.142980 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.143725 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.144075 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.144078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.144430 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:55.144642 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.144645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.145693 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.145694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.146448 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.146451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.146860 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.146861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.146973 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.147677 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:55.147689 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:55.151415 (+ 3726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:55.154496 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.154497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.154498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.154738 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.155700 (+ 962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:55.155701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:55.155702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:55.155771 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:55.156129 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:55.156130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:55.156231 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.156327 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.156454 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":122,"threads_started":1}
I20260810 07:19:55.157122 25795 ts_manager.cc:194] Registered new tserver with Master: e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577)
I20260810 07:19:55.157231 26796 heartbeater.cc:347] Connected to a master server at 127.14.164.60:39157
I20260810 07:19:55.157272 26796 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:55.157333 26796 heartbeater.cc:511] Master 127.14.164.60:39157 requested a full tablet report, sending...
I20260810 07:19:55.157462 24819 ts_manager.cc:194] Re-registered known tserver with Master: e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577)
I20260810 07:19:55.157927 24819 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:36485
I20260810 07:19:55.157970 26081 ts_manager.cc:194] Registered new tserver with Master: e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577)
I20260810 07:19:55.160754 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.149624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.14.164.61:46579)
0810 07:19:55.149748 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.149750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.150007 (+ 257us) server_negotiation.cc:407] Connection header received
0810 07:19:55.150442 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.150443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.150459 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.150526 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.151406 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.152283 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.153541 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.153912 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.153961 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.157231 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.157233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.157233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.157241 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.159197 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.159617 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.159618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.159619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.159692 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.159920 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.159921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.159921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.160097 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.160259 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.160506 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.160608 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260810 07:19:55.162709 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.149422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.14.164.62:39011)
0810 07:19:55.149814 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.149816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.150128 (+ 312us) server_negotiation.cc:407] Connection header received
0810 07:19:55.150237 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.150238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.150252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.150314 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.152664 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.153535 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.157363 (+ 3828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.157734 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.157784 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.159695 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.159696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.159697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.159703 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.161537 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.161902 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.161903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.161904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.161965 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.162187 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.162188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.162188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.162305 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.162419 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.162521 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.162606 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":119,"threads_started":1}
I20260810 07:19:55.166628 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.149892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.14.164.60:39157)
0810 07:19:55.156696 (+ 6804us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.156698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.158644 (+ 1946us) server_negotiation.cc:407] Connection header received
0810 07:19:55.159565 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.159566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.159579 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.159636 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.160347 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.161182 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.161932 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.162160 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.162244 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.163620 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.163622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.163622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.163630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.165272 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.165661 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.165662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.165663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.165731 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.165920 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.165921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.165922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.166059 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.166169 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.166442 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.166530 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6788,"thread_start_us":70,"threads_started":1}
I20260810 07:19:55.164631 24819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:55.167316 24819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:55.168179 26041 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: NON_VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: true } } }
I20260810 07:19:55.168299 26041 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:55.169553 25734 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: attempting to promote NON_VOTER ab7dc73389884a5aa28a857c270f4f12 to VOTER
I20260810 07:19:55.169691 25734 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:55.170415 26106 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 LEARNER]: Refusing update from remote peer b6e26534bc5a42bf8955212b6e8295c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 07:19:55.170513 25808 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Refusing update from remote peer b6e26534bc5a42bf8955212b6e8295c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 07:19:55.170743 26041 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 07:19:55.170843 25731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 07:19:55.171512 25731 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.171649 25808 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.171705 26106 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.172559 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.172633 26041 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.172694 26041 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:55.172670 26032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.172773 26041 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.172819 26032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:55.172833 26041 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:55.172857 26153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e26534bc5a42bf8955212b6e8295c8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:55.172963 26153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:55.172640 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:55.178104 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.174083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:37923 (local address 127.0.0.1:56406)
0810 07:19:55.174228 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.174237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.174302 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.174805 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.174806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.174808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:55.174858 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.174860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.175850 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.175851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.176951 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.176954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.177833 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.177834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.177895 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.177900 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.177949 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.177995 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260810 07:19:55.178278 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.173810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:42947 (local address 127.0.0.1:41828)
0810 07:19:55.173858 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.173869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.173907 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.174560 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.174561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.174562 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:55.174622 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.174623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.176220 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.176222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.177069 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.177072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.178059 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.178061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.178124 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.178128 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.178160 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.178206 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:19:55.178395 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.174362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.14.164.1:37923)
0810 07:19:55.174554 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.174558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.174568 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:55.174607 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.174609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.174626 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.174708 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:55.174940 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.175737 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.177107 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.177730 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.177829 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.178216 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.178265 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.178307 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260810 07:19:55.178404 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.173884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45577 (local address 127.0.0.1:40488)
0810 07:19:55.174026 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.174037 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.174086 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.174858 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.174859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.174861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:55.174906 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.174908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.176199 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.176200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.177308 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.177310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.178218 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.178219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.178275 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.178278 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.178308 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.178350 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":29,"thread_start_us":97,"threads_started":1}
I20260810 07:19:55.178814 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.173983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40488 (local address 127.14.164.3:45577)
0810 07:19:55.174682 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.174686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.174695 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:55.174732 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.174734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.174749 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.174858 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:55.174999 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.176103 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.177396 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.178128 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.178509 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.178607 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.178642 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.178699 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":104,"threads_started":1}
I20260810 07:19:55.178941 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.173995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.14.164.2:42947)
0810 07:19:55.174365 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.174370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.174380 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:55.174415 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.174417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.174431 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.174543 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:55.174772 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.176110 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.177163 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.177952 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.178365 (+ 413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.178770 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.178800 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.178850 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":88,"threads_started":1}
I20260810 07:19:55.179670 26695 tablet_service.cc:1467] Processing CreateTablet for tablet 9360e14fc9c34c8c937974dbedbb55b6 (DEFAULT_TABLE table=test-table [id=b7b0e1a84fa04993b96780d7ba174e56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:55.179666 26280 tablet_service.cc:1467] Processing CreateTablet for tablet 9360e14fc9c34c8c937974dbedbb55b6 (DEFAULT_TABLE table=test-table [id=b7b0e1a84fa04993b96780d7ba174e56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:55.179769 26488 tablet_service.cc:1467] Processing CreateTablet for tablet 9360e14fc9c34c8c937974dbedbb55b6 (DEFAULT_TABLE table=test-table [id=b7b0e1a84fa04993b96780d7ba174e56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:55.180063 26280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9360e14fc9c34c8c937974dbedbb55b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:55.180063 26695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9360e14fc9c34c8c937974dbedbb55b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:55.180145 26488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9360e14fc9c34c8c937974dbedbb55b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:55.183312 26842 tablet_bootstrap.cc:492] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Bootstrap starting.
I20260810 07:19:55.183734 26843 tablet_bootstrap.cc:492] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Bootstrap starting.
I20260810 07:19:55.184161 26842 tablet_bootstrap.cc:654] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:55.184383 26843 tablet_bootstrap.cc:654] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:55.184667 26842 log.cc:824] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:55.184818 26843 log.cc:824] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:55.185112 26846 tablet_bootstrap.cc:492] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Bootstrap starting.
I20260810 07:19:55.185581 26842 tablet_bootstrap.cc:492] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: No bootstrap required, opened a new log
I20260810 07:19:55.185581 26843 tablet_bootstrap.cc:492] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: No bootstrap required, opened a new log
I20260810 07:19:55.185658 26843 ts_tablet_manager.cc:1397] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:19:55.185658 26842 ts_tablet_manager.cc:1397] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:55.185714 26846 tablet_bootstrap.cc:654] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:55.186071 26846 log.cc:824] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:55.186889 26846 tablet_bootstrap.cc:492] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: No bootstrap required, opened a new log
I20260810 07:19:55.186973 26846 ts_tablet_manager.cc:1397] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:55.187922 26842 raft_consensus.cc:348] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.188138 26842 raft_consensus.cc:374] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:55.188177 26842 raft_consensus.cc:729] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d8d3fb6a0164924bfba34125b1747f9, State: Initialized, Role: FOLLOWER
I20260810 07:19:55.188336 26842 consensus_queue.cc:260] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.188592 26843 raft_consensus.cc:348] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.188766 26843 raft_consensus.cc:374] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:55.188848 26843 raft_consensus.cc:729] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a91f9597b843e398d0a46954830499, State: Initialized, Role: FOLLOWER
I20260810 07:19:55.188891 26588 heartbeater.cc:503] Master 127.14.164.62:39011 was elected leader, sending a full tablet report...
I20260810 07:19:55.188977 26843 consensus_queue.cc:260] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.189262 26843 ts_tablet_manager.cc:1428] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260810 07:19:55.189581 26797 heartbeater.cc:503] Master 127.14.164.62:39011 was elected leader, sending a full tablet report...
I20260810 07:19:55.190124 26842 ts_tablet_manager.cc:1428] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:19:55.190131 26846 raft_consensus.cc:348] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.190430 26846 raft_consensus.cc:374] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:55.190564 26846 raft_consensus.cc:729] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00791d14d1274bbf82945f94a7ae54c6, State: Initialized, Role: FOLLOWER
I20260810 07:19:55.190709 26846 consensus_queue.cc:260] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.190996 26846 ts_tablet_manager.cc:1428] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:19:55.191113 26376 heartbeater.cc:503] Master 127.14.164.62:39011 was elected leader, sending a full tablet report...
I20260810 07:19:55.226473 26853 raft_consensus.cc:482] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:55.226603 26853 raft_consensus.cc:504] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.227224 26853 leader_election.cc:288] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577), 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923)
I20260810 07:19:55.230538 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.227681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:37089 (local address 127.14.164.1:37923)
0810 07:19:55.227744 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.227746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.227755 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:55.227821 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.227822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.227838 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.227912 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:55.228078 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.228555 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.229646 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.230184 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.230256 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.230266 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.230425 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.230461 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:19:55.230540 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.227632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:37923 (local address 127.14.164.2:37089)
0810 07:19:55.227685 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.227695 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.227758 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.227925 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.227926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.227927 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:55.228001 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.228003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.228648 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.228649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.229532 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.229535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.230254 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.230255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.230340 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.230344 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.230416 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.230461 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":4}
I20260810 07:19:55.230650 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.227547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:58793 (local address 127.14.164.3:45577)
0810 07:19:55.227631 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.227633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.227645 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:55.227740 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.227741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.227752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.227853 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:55.228024 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.228599 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.229689 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.230257 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.230357 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.230367 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.230542 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.230585 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:19:55.230661 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.227466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45577 (local address 127.14.164.2:58793)
0810 07:19:55.227530 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.227542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.227633 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.227860 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.227861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.227863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:55.227943 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.227945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.228754 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.228755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.229582 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.229585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.230357 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.230358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.230451 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.230454 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.230532 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.230573 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":1}
I20260810 07:19:55.231010 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9360e14fc9c34c8c937974dbedbb55b6" candidate_uuid: "9d8d3fb6a0164924bfba34125b1747f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00791d14d1274bbf82945f94a7ae54c6" is_pre_election: true
I20260810 07:19:55.231000 26723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9360e14fc9c34c8c937974dbedbb55b6" candidate_uuid: "9d8d3fb6a0164924bfba34125b1747f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a91f9597b843e398d0a46954830499" is_pre_election: true
I20260810 07:19:55.231179 26311 raft_consensus.cc:2454] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d8d3fb6a0164924bfba34125b1747f9 in term 0.
I20260810 07:19:55.231179 26723 raft_consensus.cc:2454] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d8d3fb6a0164924bfba34125b1747f9 in term 0.
I20260810 07:19:55.231413 26412 leader_election.cc:302] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00791d14d1274bbf82945f94a7ae54c6, 9d8d3fb6a0164924bfba34125b1747f9; no voters:
I20260810 07:19:55.231554 26853 raft_consensus.cc:2781] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:55.231614 26853 raft_consensus.cc:482] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:55.231640 26853 raft_consensus.cc:3037] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:55.232393 26853 raft_consensus.cc:504] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.232563 26853 leader_election.cc:288] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [CANDIDATE]: Term 1 election: Requested vote from peers e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577), 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923)
I20260810 07:19:55.232792 26723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9360e14fc9c34c8c937974dbedbb55b6" candidate_uuid: "9d8d3fb6a0164924bfba34125b1747f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a91f9597b843e398d0a46954830499"
I20260810 07:19:55.232932 26723 raft_consensus.cc:3037] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:55.233052 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9360e14fc9c34c8c937974dbedbb55b6" candidate_uuid: "9d8d3fb6a0164924bfba34125b1747f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00791d14d1274bbf82945f94a7ae54c6"
I20260810 07:19:55.233126 26311 raft_consensus.cc:3037] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:55.233843 26723 raft_consensus.cc:2454] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d8d3fb6a0164924bfba34125b1747f9 in term 1.
I20260810 07:19:55.233901 26311 raft_consensus.cc:2454] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d8d3fb6a0164924bfba34125b1747f9 in term 1.
I20260810 07:19:55.234058 26412 leader_election.cc:302] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d8d3fb6a0164924bfba34125b1747f9, e6a91f9597b843e398d0a46954830499; no voters:
I20260810 07:19:55.234194 26853 raft_consensus.cc:2781] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:55.234366 26853 raft_consensus.cc:686] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 LEADER]: Becoming Leader. State: Replica: 9d8d3fb6a0164924bfba34125b1747f9, State: Running, Role: LEADER
I20260810 07:19:55.234488 26853 consensus_queue.cc:237] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } }
I20260810 07:19:55.235422 24819 catalog_manager.cc:5495] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2). New cstate: current_term: 1 leader_uuid: "9d8d3fb6a0164924bfba34125b1747f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } health_report { overall_health: UNKNOWN } } }
W20260810 07:19:55.243382 26591 tablet.cc:2369] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:55.249950 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.247397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.14.164.2:42947)
0810 07:19:55.247462 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.247464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.247473 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:55.247530 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.247531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.247549 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.247611 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:55.247775 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.248369 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.249238 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.249428 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.249479 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.249610 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:55.249712 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:55.249775 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.249807 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.249827 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:55.252036 26723 raft_consensus.cc:1261] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Refusing update from remote peer 9d8d3fb6a0164924bfba34125b1747f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:55.252044 26311 raft_consensus.cc:1261] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 FOLLOWER]: Refusing update from remote peer 9d8d3fb6a0164924bfba34125b1747f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:55.252413 26860 consensus_queue.cc:1035] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:55.252563 26853 consensus_queue.cc:1035] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:55.255913 26868 mvcc.cc:205] Tried to move back new op lower bound from 7316874834948960256 to 7316874834881286144. Current Snapshot: MvccSnapshot[applied={T|T < 7316874834948960256}]
I20260810 07:19:55.256136 26867 mvcc.cc:205] Tried to move back new op lower bound from 7316874834948960256 to 7316874834881286144. Current Snapshot: MvccSnapshot[applied={T|T < 7316874834948960256}]
I20260810 07:19:55.258430 24819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:55.258530 24819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:55.259842 24819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e26534bc5a42bf8955212b6e8295c8: Sending DeleteTablet for 3 replicas of tablet 9360e14fc9c34c8c937974dbedbb55b6
I20260810 07:19:55.260221 26280 tablet_service.cc:1514] Processing DeleteTablet for tablet 9360e14fc9c34c8c937974dbedbb55b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:55 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:56406
I20260810 07:19:55.260294 26488 tablet_service.cc:1514] Processing DeleteTablet for tablet 9360e14fc9c34c8c937974dbedbb55b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:55 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:41828
I20260810 07:19:55.260494 26695 tablet_service.cc:1514] Processing DeleteTablet for tablet 9360e14fc9c34c8c937974dbedbb55b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:55 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:40488
I20260810 07:19:55.261108 26885 tablet_replica.cc:330] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: stopping tablet replica
I20260810 07:19:55.261300 26884 tablet_replica.cc:330] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: stopping tablet replica
I20260810 07:19:55.261402 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 24773
I20260810 07:19:55.261415 26884 raft_consensus.cc:2229] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:55.268874 26887 tablet_replica.cc:330] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: stopping tablet replica
I20260810 07:19:55.269153 26887 raft_consensus.cc:2229] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:55.261552 26884 raft_consensus.cc:2258] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:55.269335 26887 raft_consensus.cc:2258] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:55.272630 26884 ts_tablet_manager.cc:1905] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:55.273538 26887 ts_tablet_manager.cc:1905] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:55.274647 26884 ts_tablet_manager.cc:1918] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:55.274762 26884 log.cc:1196] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/9360e14fc9c34c8c937974dbedbb55b6
I20260810 07:19:55.275084 26884 ts_tablet_manager.cc:1939] T 9360e14fc9c34c8c937974dbedbb55b6 P 00791d14d1274bbf82945f94a7ae54c6: Deleting consensus metadata
I20260810 07:19:55.275688 26887 ts_tablet_manager.cc:1918] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:55.275790 26887 log.cc:1196] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/9360e14fc9c34c8c937974dbedbb55b6
I20260810 07:19:55.276043 26887 ts_tablet_manager.cc:1939] T 9360e14fc9c34c8c937974dbedbb55b6 P e6a91f9597b843e398d0a46954830499: Deleting consensus metadata
I20260810 07:19:55.277000 26885 raft_consensus.cc:2229] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:55.277179 26885 raft_consensus.cc:2258] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:55.277817 26885 ts_tablet_manager.cc:1905] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:55.279614 26885 ts_tablet_manager.cc:1918] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:55.279711 26885 log.cc:1196] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/9360e14fc9c34c8c937974dbedbb55b6
I20260810 07:19:55.279989 26885 ts_tablet_manager.cc:1939] T 9360e14fc9c34c8c937974dbedbb55b6 P 9d8d3fb6a0164924bfba34125b1747f9: Deleting consensus metadata
W20260810 07:19:55.283550 26202 connection.cc:535] server connection from 127.0.0.1:56406 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:19:55.283565 26425 connection.cc:535] server connection from 127.0.0.1:41828 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:19:55.283787 26627 connection.cc:535] server connection from 127.0.0.1:40488 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:19:55.285416 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.261230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.14.164.60:39157)
0810 07:19:55.261273 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.261275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.277464 (+ 16189us) server_negotiation.cc:407] Connection header received
0810 07:19:55.277593 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.277594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.277608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.277642 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.278537 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.279298 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.280397 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.280644 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.280682 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.282290 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.282291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.282292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.282299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.284000 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.284544 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.284545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.284546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.284612 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.284859 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.284860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.284860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.284985 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.285092 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.285211 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.285296 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:19:55.292937 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.286119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.14.164.60:39157)
0810 07:19:55.286160 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.286162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.286180 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:19:55.286198 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.286199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.286207 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.286256 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.286968 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.287439 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.288168 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.288371 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.288451 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.289950 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.289951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.289952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.289959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.291615 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.292013 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.292015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.292015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.292083 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.292311 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.292312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.292313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.292471 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.292584 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.292704 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.292805 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:55.294051 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.286328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.14.164.61:46579)
0810 07:19:55.286386 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.286388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.286516 (+ 128us) server_negotiation.cc:407] Connection header received
0810 07:19:55.286564 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.286565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.286581 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.286637 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.287322 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.288108 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.289214 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.289578 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.289666 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.291052 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.291053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.291054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.291062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.292865 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.293251 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.293252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.293252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.293319 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.293537 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.293538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.293539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.293674 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.293780 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.293850 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.293951 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:19:56.257292 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.256964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:50377)
0810 07:19:56.257124 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.257225 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260810 07:19:56.257386 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.257116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:55701)
0810 07:19:56.257226 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.257326 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
W20260810 07:19:56.257578 26797 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:39011 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
W20260810 07:19:56.257687 26376 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:39011 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.258314 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.257887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:52009)
0810 07:19:56.258186 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.258260 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":83,"threads_started":1}
W20260810 07:19:56.258478 26588 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:39011 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.760622 26903 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b6e26534bc5a42bf8955212b6e8295c8)
I20260810 07:19:56.760787 26904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b6e26534bc5a42bf8955212b6e8295c8)
I20260810 07:19:56.760895 26903 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:56.760932 26904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:56.761207 26904 leader_election.cc:288] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011), ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60:39157)
I20260810 07:19:56.761217 26903 leader_election.cc:288] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011), 44c5429b0e6843d188c47d116833847f (127.14.164.61:46579)
I20260810 07:19:56.761605 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.761362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53942)
0810 07:19:56.761495 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.761559 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
W20260810 07:19:56.761791 26072 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.761621 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.761354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53940)
0810 07:19:56.761535 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.761582 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":56,"threads_started":1}
W20260810 07:19:56.762174 25771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.762598 26907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.762483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53958)
0810 07:19:56.762527 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.762564 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:19:56.762693 26072 leader_election.cc:334] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.763047 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.762931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53966)
0810 07:19:56.762984 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.763018 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:19:56.763190 25771 leader_election.cc:334] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.764706 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.761569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.14.164.61:46579)
0810 07:19:56.761729 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:56.761732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:56.761742 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:56.761860 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:56.761861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:56.761873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:56.761956 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:56.762143 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.762801 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.763965 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.764344 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.764443 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:56.764456 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:56.764606 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 07:19:56.764645 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":43,"threads_started":1}
I20260810 07:19:56.764708 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.761439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:46579 (local address 127.0.0.1:56736)
0810 07:19:56.761658 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.761672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:56.761744 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:56.761973 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:56.761975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:56.761979 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:56.762049 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:56.762051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.762899 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.762900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:56.763851 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:56.763854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.764446 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.764447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:56.764515 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:56.764524 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 07:19:56.764597 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 07:19:56.764641 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":42,"threads_started":1}
I20260810 07:19:56.765030 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.761513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:39157 (local address 127.0.0.1:39500)
0810 07:19:56.761767 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.761777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:56.761835 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:56.762257 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:56.762257 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:56.762259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:56.762295 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:56.762297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.763320 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.763321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:56.764133 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:56.764135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.764754 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.764755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:56.764830 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:56.764835 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:56.764891 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:19:56.764935 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":33,"threads_started":1}
I20260810 07:19:56.765115 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.761654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.14.164.60:39157)
0810 07:19:56.761825 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:56.761827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:56.761835 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:56.761990 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:56.761991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:56.762001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:56.762069 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:56.762375 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.763218 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.764230 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.764646 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.764754 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:56.764768 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:56.764989 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:19:56.765037 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":154,"thread_start_us":61,"threads_started":1}
I20260810 07:19:56.765034 25808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab7dc73389884a5aa28a857c270f4f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44c5429b0e6843d188c47d116833847f" is_pre_election: true
I20260810 07:19:56.765197 25808 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab7dc73389884a5aa28a857c270f4f12 in term 1.
I20260810 07:19:56.765309 26106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c5429b0e6843d188c47d116833847f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab7dc73389884a5aa28a857c270f4f12" is_pre_election: true
I20260810 07:19:56.765403 26106 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c5429b0e6843d188c47d116833847f in term 1.
I20260810 07:19:56.765553 25771 leader_election.cc:302] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44c5429b0e6843d188c47d116833847f, ab7dc73389884a5aa28a857c270f4f12; no voters: b6e26534bc5a42bf8955212b6e8295c8
I20260810 07:19:56.765563 26070 leader_election.cc:302] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44c5429b0e6843d188c47d116833847f, ab7dc73389884a5aa28a857c270f4f12; no voters: b6e26534bc5a42bf8955212b6e8295c8
I20260810 07:19:56.765682 26903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:19:56.765751 26903 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b6e26534bc5a42bf8955212b6e8295c8)
I20260810 07:19:56.765851 26903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:19:56.766662 26903 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:56.766835 26903 leader_election.cc:288] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [CANDIDATE]: Term 2 election: Requested vote from peers b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011), 44c5429b0e6843d188c47d116833847f (127.14.164.61:46579)
I20260810 07:19:56.766947 26904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:19:56.767004 26904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Starting leader election (detected failure of leader b6e26534bc5a42bf8955212b6e8295c8)
I20260810 07:19:56.767027 26904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:19:56.767050 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.766910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53972)
0810 07:19:56.766981 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.767018 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:19:56.767108 25808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab7dc73389884a5aa28a857c270f4f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44c5429b0e6843d188c47d116833847f"
I20260810 07:19:56.767445 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.767328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53978)
0810 07:19:56.767384 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.767418 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:19:56.767544 26072 leader_election.cc:334] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.767922 26904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:56.768070 26904 leader_election.cc:288] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 2 election: Requested vote from peers b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011), ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60:39157)
I20260810 07:19:56.768077 25808 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ab7dc73389884a5aa28a857c270f4f12 in current term 2: Already voted for candidate 44c5429b0e6843d188c47d116833847f in this term.
I20260810 07:19:56.768325 26070 leader_election.cc:302] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab7dc73389884a5aa28a857c270f4f12; no voters: 44c5429b0e6843d188c47d116833847f, b6e26534bc5a42bf8955212b6e8295c8
I20260810 07:19:56.768344 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.768156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53982)
0810 07:19:56.768288 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.768316 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 07:19:56.768471 26106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c5429b0e6843d188c47d116833847f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab7dc73389884a5aa28a857c270f4f12"
I20260810 07:19:56.768503 26903 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:19:56.768576 26106 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 44c5429b0e6843d188c47d116833847f in current term 2: Already voted for candidate ab7dc73389884a5aa28a857c270f4f12 in this term.
I20260810 07:19:56.768787 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.768653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:53988)
0810 07:19:56.768692 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:56.768763 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260810 07:19:56.768877 25771 leader_election.cc:334] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:56.768922 25771 leader_election.cc:302] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44c5429b0e6843d188c47d116833847f; no voters: ab7dc73389884a5aa28a857c270f4f12, b6e26534bc5a42bf8955212b6e8295c8
I20260810 07:19:56.768992 26904 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:19:57.258384 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.258092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:41877)
0810 07:19:57.258257 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.258322 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260810 07:19:57.258400 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.258092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:36205)
0810 07:19:57.258258 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.258313 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260810 07:19:57.259039 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.258816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:57743)
0810 07:19:57.258951 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.258988 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260810 07:19:58.259208 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.258940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:35537)
0810 07:19:58.259095 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.259141 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260810 07:19:58.259581 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.258944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:43739)
0810 07:19:58.259138 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.259194 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260810 07:19:58.259652 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.259440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:46685)
0810 07:19:58.259564 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.259604 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260810 07:19:58.389668 26927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:58.389777 26927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:58.389920 26927 leader_election.cc:288] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011), ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60:39157)
I20260810 07:19:58.390295 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.390092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:54242)
0810 07:19:58.390197 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.390243 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":58,"threads_started":1}
I20260810 07:19:58.390383 26106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c5429b0e6843d188c47d116833847f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab7dc73389884a5aa28a857c270f4f12" is_pre_election: true
I20260810 07:19:58.390491 26106 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c5429b0e6843d188c47d116833847f in term 2.
I20260810 07:19:58.390759 25771 leader_election.cc:302] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c5429b0e6843d188c47d116833847f, ab7dc73389884a5aa28a857c270f4f12; no voters:
I20260810 07:19:58.390820 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.390692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:54246)
0810 07:19:58.390745 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.390795 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 07:19:58.390864 26927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 07:19:58.390894 26927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:58.390923 26927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 2 FOLLOWER]: Advancing to term 3
W20260810 07:19:58.391491 25771 leader_election.cc:334] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:58.391741 26927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:58.391880 26927 leader_election.cc:288] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 3 election: Requested vote from peers b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011), ab7dc73389884a5aa28a857c270f4f12 (127.14.164.60:39157)
I20260810 07:19:58.392072 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.391995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:54256)
0810 07:19:58.392027 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.392054 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 07:19:58.392164 26106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c5429b0e6843d188c47d116833847f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab7dc73389884a5aa28a857c270f4f12"
I20260810 07:19:58.392244 26106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:19:58.393033 26106 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c5429b0e6843d188c47d116833847f in term 3.
I20260810 07:19:58.393043 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.392955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:54264)
0810 07:19:58.392998 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.393021 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 07:19:58.393115 25771 leader_election.cc:334] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
I20260810 07:19:58.393180 25771 leader_election.cc:302] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44c5429b0e6843d188c47d116833847f, ab7dc73389884a5aa28a857c270f4f12; no voters: b6e26534bc5a42bf8955212b6e8295c8
I20260810 07:19:58.393247 26927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:19:58.393368 26927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [term 3 LEADER]: Becoming Leader. State: Replica: 44c5429b0e6843d188c47d116833847f, State: Running, Role: LEADER
I20260810 07:19:58.393446 26927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } }
I20260810 07:19:58.393836 26932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c5429b0e6843d188c47d116833847f. Latest consensus state: current_term: 3 leader_uuid: "44c5429b0e6843d188c47d116833847f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:58.393949 26932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:58.394138 26934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:58.394542 26934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:58.394603 26934 catalog_manager.cc:1259] Loaded cluster ID: ef8aacfb45514870bf4025d483ab3a5f
I20260810 07:19:58.394650 26934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:58.394845 26934 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:58.394917 26934 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Loaded TSK: 0
I20260810 07:19:58.395064 26934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:58.463037 25794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:58.464910 25794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:58.466185 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.465993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:54284)
0810 07:19:58.466101 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.466147 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 07:19:58.466264 26106 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [term 3 FOLLOWER]: Refusing update from remote peer 44c5429b0e6843d188c47d116833847f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 07:19:58.466666 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.466552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.0.0.1:54298)
0810 07:19:58.466605 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.466636 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 07:19:58.466677 26932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 07:19:58.466921 25771 consensus_peers.cc:487] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f -> Peer b6e26534bc5a42bf8955212b6e8295c8 (127.14.164.62:39011): Couldn't send request to peer b6e26534bc5a42bf8955212b6e8295c8. Status: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:19:58.467512 26936 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c5429b0e6843d188c47d116833847f. Latest consensus state: current_term: 3 leader_uuid: "44c5429b0e6843d188c47d116833847f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:58.467635 26936 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:58.468174 26932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "44c5429b0e6843d188c47d116833847f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:58.468284 26932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:58.468278 26936 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "44c5429b0e6843d188c47d116833847f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6e26534bc5a42bf8955212b6e8295c8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39011 } } peers { permanent_uuid: "44c5429b0e6843d188c47d116833847f" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 46579 } attrs { promote: false } } peers { permanent_uuid: "ab7dc73389884a5aa28a857c270f4f12" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 39157 } attrs { promote: false } } }
I20260810 07:19:58.468369 26936 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7dc73389884a5aa28a857c270f4f12 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:58.474012 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.470798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.14.164.1:37923)
0810 07:19:58.470936 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.470938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.470947 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:58.470977 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.470978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.470990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.471103 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:58.471196 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.471726 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.473076 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.473644 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.473725 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.473740 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.473894 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.473938 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260810 07:19:58.474012 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.470268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:37923 (local address 127.0.0.1:56420)
0810 07:19:58.470430 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.470440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.470494 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.471108 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.471108 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.471110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:58.471141 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.471142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.471809 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.471810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.472975 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.472978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.473724 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.473725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.473789 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.473793 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.473884 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.473932 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260810 07:19:58.474248 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.470211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:42947 (local address 127.0.0.1:41842)
0810 07:19:58.470248 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.470260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.470332 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.470920 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.470921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.470924 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:58.470988 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.470990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.472149 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.472150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.473225 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.473228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.474001 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.474001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.474069 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.474074 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.474107 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.474160 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":193}
I20260810 07:19:58.474478 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.470475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45577 (local address 127.0.0.1:40498)
0810 07:19:58.470936 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.470946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.470990 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.471182 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.471183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.471185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:58.471241 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.471242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.472186 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.472187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.473398 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.473401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.474247 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.474248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.474307 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.474312 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.474367 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.474410 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":65,"threads_started":1}
I20260810 07:19:58.474732 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.470610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.14.164.2:42947)
0810 07:19:58.470726 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.470728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.470737 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:58.470765 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.470766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.470779 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.471057 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:58.471113 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.471898 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.473317 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.473896 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.474457 (+ 561us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.474478 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.474519 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.474572 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260810 07:19:58.474771 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.470812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.14.164.3:45577)
0810 07:19:58.470975 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.470978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.470987 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:58.471048 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.471049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.471061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.471131 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:58.471321 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.472108 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.473478 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.474170 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.474236 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.474251 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.474576 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.474692 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":68,"threads_started":1}
I20260810 07:19:58.474750 26280 tablet_service.cc:1467] Processing CreateTablet for tablet 28888929c6494f3c9a37eb72ed2bc621 (DEFAULT_TABLE table=test-table [id=5fb03765875a485eb6c3536c9703936b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:58.474923 26280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28888929c6494f3c9a37eb72ed2bc621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:58.475090 26488 tablet_service.cc:1467] Processing CreateTablet for tablet 28888929c6494f3c9a37eb72ed2bc621 (DEFAULT_TABLE table=test-table [id=5fb03765875a485eb6c3536c9703936b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:58.475208 26488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28888929c6494f3c9a37eb72ed2bc621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:58.476555 26955 tablet_bootstrap.cc:492] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6: Bootstrap starting.
I20260810 07:19:58.476672 26695 tablet_service.cc:1467] Processing CreateTablet for tablet 28888929c6494f3c9a37eb72ed2bc621 (DEFAULT_TABLE table=test-table [id=5fb03765875a485eb6c3536c9703936b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:58.476840 26695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28888929c6494f3c9a37eb72ed2bc621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:58.476824 26957 tablet_bootstrap.cc:492] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Bootstrap starting.
I20260810 07:19:58.477087 26955 tablet_bootstrap.cc:654] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:58.477226 26957 tablet_bootstrap.cc:654] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:58.477800 26955 tablet_bootstrap.cc:492] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6: No bootstrap required, opened a new log
I20260810 07:19:58.477905 26955 ts_tablet_manager.cc:1397] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:58.477945 26957 tablet_bootstrap.cc:492] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: No bootstrap required, opened a new log
I20260810 07:19:58.477991 26957 ts_tablet_manager.cc:1397] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:58.478048 26955 raft_consensus.cc:348] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.478120 26955 raft_consensus.cc:374] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:58.478086 26957 raft_consensus.cc:348] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.478142 26955 raft_consensus.cc:729] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00791d14d1274bbf82945f94a7ae54c6, State: Initialized, Role: FOLLOWER
I20260810 07:19:58.478147 26957 raft_consensus.cc:374] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:58.478142 26963 tablet_bootstrap.cc:492] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Bootstrap starting.
I20260810 07:19:58.478169 26957 raft_consensus.cc:729] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d8d3fb6a0164924bfba34125b1747f9, State: Initialized, Role: FOLLOWER
I20260810 07:19:58.478214 26955 consensus_queue.cc:260] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.478235 26957 consensus_queue.cc:260] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.478439 26957 ts_tablet_manager.cc:1428] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:19:58.478439 26955 ts_tablet_manager.cc:1428] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:19:58.478505 26963 tablet_bootstrap.cc:654] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:58.478914 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.478795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:40259)
0810 07:19:58.478840 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.478880 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 07:19:58.479133 26963 tablet_bootstrap.cc:492] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: No bootstrap required, opened a new log
I20260810 07:19:58.479184 26963 ts_tablet_manager.cc:1397] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:58.479337 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.479190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:47123)
0810 07:19:58.479266 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.479303 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 07:19:58.479364 26963 raft_consensus.cc:348] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.479447 26963 raft_consensus.cc:374] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:58.479480 26963 raft_consensus.cc:729] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a91f9597b843e398d0a46954830499, State: Initialized, Role: FOLLOWER
I20260810 07:19:58.479532 26963 consensus_queue.cc:260] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.479723 26963 ts_tablet_manager.cc:1428] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:58.480065 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.479939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:44753)
0810 07:19:58.480005 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.480032 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:19:58.480142 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.479992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:35109)
0810 07:19:58.480094 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.480119 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 07:19:58.480548 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.480457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:47401)
0810 07:19:58.480491 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.480523 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 07:19:58.480892 26590 heartbeater.cc:503] Master 127.14.164.61:46579 was elected leader, sending a full tablet report...
I20260810 07:19:58.481177 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.481029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:46629)
0810 07:19:58.481116 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.481149 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 07:19:58.591638 26380 tablet.cc:2369] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:58.632009 26965 raft_consensus.cc:482] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:58.632110 26965 raft_consensus.cc:504] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.632454 26965 leader_election.cc:288] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947), e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577)
I20260810 07:19:58.635437 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.632789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37137 (local address 127.14.164.3:45577)
0810 07:19:58.632835 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.632838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.632848 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:58.632911 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.632911 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.632922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.632987 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:58.633172 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.633672 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.634702 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.635071 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.635155 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.635166 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.635323 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.635374 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:19:58.635452 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.632650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45577 (local address 127.14.164.1:37137)
0810 07:19:58.632710 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.632722 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.632796 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.633010 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.633011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.633014 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:58.633095 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.633096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.633758 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.633760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.634604 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.634607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.635156 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.635157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.635235 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.635239 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.635313 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.635353 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:19:58.635799 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.632836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:50505 (local address 127.14.164.2:42947)
0810 07:19:58.632901 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.632904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.632991 (+ 87us) server_negotiation.cc:407] Connection header received
0810 07:19:58.633071 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.633072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.633084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.633163 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:58.633357 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.633881 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.634949 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.635358 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.635445 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.635456 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.635663 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.635713 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:58.635748 26723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28888929c6494f3c9a37eb72ed2bc621" candidate_uuid: "00791d14d1274bbf82945f94a7ae54c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a91f9597b843e398d0a46954830499" is_pre_election: true
I20260810 07:19:58.635865 26723 raft_consensus.cc:2454] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00791d14d1274bbf82945f94a7ae54c6 in term 0.
I20260810 07:19:58.636067 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.632664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:42947 (local address 127.14.164.1:50505)
0810 07:19:58.632899 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.632911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.633003 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.633166 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.633167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.633169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:58.633270 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.633272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.633973 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.633974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.634846 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.634849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.635463 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.635464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.635552 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.635557 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.635900 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.635958 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":146,"threads_started":1}
I20260810 07:19:58.636051 26202 leader_election.cc:302] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00791d14d1274bbf82945f94a7ae54c6, e6a91f9597b843e398d0a46954830499; no voters:
I20260810 07:19:58.636231 26965 raft_consensus.cc:2781] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:58.636278 26965 raft_consensus.cc:482] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:58.636305 26965 raft_consensus.cc:3037] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:58.636379 26516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28888929c6494f3c9a37eb72ed2bc621" candidate_uuid: "00791d14d1274bbf82945f94a7ae54c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d8d3fb6a0164924bfba34125b1747f9" is_pre_election: true
I20260810 07:19:58.636518 26516 raft_consensus.cc:2454] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00791d14d1274bbf82945f94a7ae54c6 in term 0.
I20260810 07:19:58.637284 26965 raft_consensus.cc:504] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.637503 26965 leader_election.cc:288] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947), e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577)
I20260810 07:19:58.637652 26516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28888929c6494f3c9a37eb72ed2bc621" candidate_uuid: "00791d14d1274bbf82945f94a7ae54c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d8d3fb6a0164924bfba34125b1747f9"
I20260810 07:19:58.637802 26516 raft_consensus.cc:3037] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:58.637750 26723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28888929c6494f3c9a37eb72ed2bc621" candidate_uuid: "00791d14d1274bbf82945f94a7ae54c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a91f9597b843e398d0a46954830499"
I20260810 07:19:58.637873 26723 raft_consensus.cc:3037] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:58.638729 26723 raft_consensus.cc:2454] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00791d14d1274bbf82945f94a7ae54c6 in term 1.
I20260810 07:19:58.638753 26516 raft_consensus.cc:2454] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00791d14d1274bbf82945f94a7ae54c6 in term 1.
I20260810 07:19:58.638968 26204 leader_election.cc:302] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00791d14d1274bbf82945f94a7ae54c6, 9d8d3fb6a0164924bfba34125b1747f9; no voters:
I20260810 07:19:58.639083 26965 raft_consensus.cc:2781] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:58.639282 26965 raft_consensus.cc:686] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [term 1 LEADER]: Becoming Leader. State: Replica: 00791d14d1274bbf82945f94a7ae54c6, State: Running, Role: LEADER
I20260810 07:19:58.639472 26377 heartbeater.cc:503] Master 127.14.164.61:46579 was elected leader, sending a full tablet report...
I20260810 07:19:58.639669 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.639540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:56377)
0810 07:19:58.639586 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.639616 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:19:58.639622 26965 consensus_queue.cc:237] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } }
I20260810 07:19:58.640491 25794 catalog_manager.cc:5495] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "00791d14d1274bbf82945f94a7ae54c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00791d14d1274bbf82945f94a7ae54c6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 } health_report { overall_health: UNKNOWN } } }
W20260810 07:19:58.645593 26800 tablet.cc:2369] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:58.655912 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.653316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.14.164.1:37923)
0810 07:19:58.653383 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.653385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.653395 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:58.653454 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.653455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.653476 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.653547 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:58.653636 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.654205 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.654980 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.655204 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.655268 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.655468 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:58.655585 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:58.655657 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.655693 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.655721 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:19:58.657377 26723 raft_consensus.cc:1261] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Refusing update from remote peer 00791d14d1274bbf82945f94a7ae54c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:58.657507 26516 raft_consensus.cc:1261] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Refusing update from remote peer 00791d14d1274bbf82945f94a7ae54c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:58.657703 26974 consensus_queue.cc:1035] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6a91f9597b843e398d0a46954830499" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:58.657902 26965 consensus_queue.cc:1035] T 28888929c6494f3c9a37eb72ed2bc621 P 00791d14d1274bbf82945f94a7ae54c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d8d3fb6a0164924bfba34125b1747f9" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:58.658445 26798 heartbeater.cc:503] Master 127.14.164.61:46579 was elected leader, sending a full tablet report...
I20260810 07:19:58.658520 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.658372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:50577)
0810 07:19:58.658443 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.658487 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 07:19:58.659417 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.659310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:36067)
0810 07:19:58.659353 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.659383 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 07:19:58.660202 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.660101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:42189)
0810 07:19:58.660144 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.660171 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 07:19:58.660562 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.660476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.1:48601)
0810 07:19:58.660510 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.660534 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 07:19:58.660626 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.660498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.3:33055)
0810 07:19:58.660566 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.660593 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 07:19:58.661586 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.661460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39011 (local address 127.14.164.2:45431)
0810 07:19:58.661524 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.661557 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:19:58.662883 25795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:58.662992 25795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:58.664799 25795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44c5429b0e6843d188c47d116833847f: Sending DeleteTablet for 3 replicas of tablet 28888929c6494f3c9a37eb72ed2bc621
I20260810 07:19:58.665221 26488 tablet_service.cc:1514] Processing DeleteTablet for tablet 28888929c6494f3c9a37eb72ed2bc621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:58 UTC) from {username='oryx', principal='oryx/127.14.164.61@KRBTEST.COM'} at 127.0.0.1:41842
I20260810 07:19:58.665256 26695 tablet_service.cc:1514] Processing DeleteTablet for tablet 28888929c6494f3c9a37eb72ed2bc621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:58 UTC) from {username='oryx', principal='oryx/127.14.164.61@KRBTEST.COM'} at 127.0.0.1:40498
I20260810 07:19:58.665467 26993 tablet_replica.cc:330] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: stopping tablet replica
I20260810 07:19:58.665500 26280 tablet_service.cc:1514] Processing DeleteTablet for tablet 28888929c6494f3c9a37eb72ed2bc621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:58 UTC) from {username='oryx', principal='oryx/127.14.164.61@KRBTEST.COM'} at 127.0.0.1:56420
I20260810 07:19:58.665534 26993 raft_consensus.cc:2229] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:58.665586 26993 raft_consensus.cc:2258] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:58.665755 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 26024
I20260810 07:19:58.666623 26993 ts_tablet_manager.cc:1905] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:58.668709 26995 tablet_replica.cc:330] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: stopping tablet replica
I20260810 07:19:58.668710 26993 ts_tablet_manager.cc:1918] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:58.668793 26993 log.cc:1196] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/28888929c6494f3c9a37eb72ed2bc621
I20260810 07:19:58.668800 26995 raft_consensus.cc:2229] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:58.668859 26995 raft_consensus.cc:2258] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:58.669319 26993 ts_tablet_manager.cc:1939] T 28888929c6494f3c9a37eb72ed2bc621 P 9d8d3fb6a0164924bfba34125b1747f9: Deleting consensus metadata
I20260810 07:19:58.669718 26995 ts_tablet_manager.cc:1905] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:58.671446 25769 catalog_manager.cc:4841] TS 9d8d3fb6a0164924bfba34125b1747f9 (127.14.164.2:42947): tablet 28888929c6494f3c9a37eb72ed2bc621 (table test-table [id=5fb03765875a485eb6c3536c9703936b]) successfully deleted
I20260810 07:19:58.671779 26995 ts_tablet_manager.cc:1918] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:58.671919 26995 log.cc:1196] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/28888929c6494f3c9a37eb72ed2bc621
I20260810 07:19:58.672156 26995 ts_tablet_manager.cc:1939] T 28888929c6494f3c9a37eb72ed2bc621 P e6a91f9597b843e398d0a46954830499: Deleting consensus metadata
I20260810 07:19:58.673499 25768 catalog_manager.cc:4841] TS e6a91f9597b843e398d0a46954830499 (127.14.164.3:45577): tablet 28888929c6494f3c9a37eb72ed2bc621 (table test-table [id=5fb03765875a485eb6c3536c9703936b]) successfully deleted
I20260810 07:19:58.682459 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 26386
I20260810 07:19:58.682545 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.682387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:37923 (local address 127.0.0.1:56438)
0810 07:19:58.682456 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.682502 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:37923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 07:19:58.691689 25768 catalog_manager.cc:4584] TS 00791d14d1274bbf82945f94a7ae54c6 (127.14.164.1:37923): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28888929c6494f3c9a37eb72ed2bc621: Network error: Client connection negotiation failed: client connection to 127.14.164.1:37923: connect: Connection refused (error 111)
I20260810 07:19:58.697911 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 26597
I20260810 07:19:58.712994 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 25591
I20260810 07:19:58.724474 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 25877
2026-08-10T07:19:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6761 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:58 dist-test-slave-x5fp krb5kdc[27003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:58 dist-test-slave-x5fp krb5kdc[27003](info): set up 2 sockets
Aug 10 07:19:58 dist-test-slave-x5fp krb5kdc[27003](info): commencing operation
krb5kdc: starting...
W20260810 07:19:59.994419 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab.
Entry for principal oryx/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab.
I20260810 07:20:00.136790 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:33921
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab
--principal=oryx/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:33921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:00.222252 27023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:00.222482 27023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:00.222505 27023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:00.223699 27023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:00.223747 27023 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:20:00.223760 27023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:00.223773 27023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:00.223786 27023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:00.223798 27023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:00.225368 27023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:33921
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.62.keytab
--principal=oryx/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:33921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:00.225633 27023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:00.225898 27023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:00.231720 27023 server_base.cc:1034] running on GCE node
W20260810 07:20:00.231702 27033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.231690 27030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.231702 27031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:00.232244 27023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:00.232607 27023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:00.233780 27023 hybrid_clock.cc:648] HybridClock initialized: now 1786346400233766 us; error 31 us; skew 500 ppm
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:00.236562 27023 init.cc:377] Logged in from keytab as oryx/127.14.164.62@KRBTEST.COM (short username oryx)
I20260810 07:20:00.237143 27023 webserver.cc:460] Webserver started at http://127.14.164.62:39411/ using document root <none> and password file <none>
I20260810 07:20:00.237407 27023 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:00.237474 27023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:00.237634 27023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:00.238790 27023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "e0648550a65b45c3aa9626b9c15e25c9"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "9d8107746a8619138a6f1a5c5f7b0d02"
server_key_iv: "0da64dc4194693d44fabd9bd7acc321d"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.239190 27023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "e0648550a65b45c3aa9626b9c15e25c9"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "9d8107746a8619138a6f1a5c5f7b0d02"
server_key_iv: "0da64dc4194693d44fabd9bd7acc321d"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.240650 27023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:20:00.241605 27044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.241896 27023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.242017 27023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "e0648550a65b45c3aa9626b9c15e25c9"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "9d8107746a8619138a6f1a5c5f7b0d02"
server_key_iv: "0da64dc4194693d44fabd9bd7acc321d"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.242120 27023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:00.267580 27023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:00.269896 27023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:00.270071 27023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:00.275241 27023 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:33921
I20260810 07:20:00.275309 27138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:33921 every 8 connection(s)
I20260810 07:20:00.275689 27023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:00.276479 27142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.278970 27142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Bootstrap starting.
I20260810 07:20:00.279570 27142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.279836 27142 log.cc:824] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:00.280555 27142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: No bootstrap required, opened a new log
I20260810 07:20:00.282675 27142 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0648550a65b45c3aa9626b9c15e25c9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33921 } }
I20260810 07:20:00.282820 27142 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.282845 27142 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0648550a65b45c3aa9626b9c15e25c9, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.282991 27142 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0648550a65b45c3aa9626b9c15e25c9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33921 } }
I20260810 07:20:00.283073 27142 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:00.283097 27142 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:00.283131 27142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.283702 27142 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0648550a65b45c3aa9626b9c15e25c9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33921 } }
I20260810 07:20:00.283813 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 27023
I20260810 07:20:00.283831 27142 leader_election.cc:302] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0648550a65b45c3aa9626b9c15e25c9; no voters:
I20260810 07:20:00.283936 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:20:00.284029 27142 leader_election.cc:288] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:00.284114 27150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:00.284194 14992 external_mini_cluster.cc:1428] Setting key b7ab2d5e40ac3339a045307675512728
I20260810 07:20:00.284310 27142 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:00.284320 27150 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [term 1 LEADER]: Becoming Leader. State: Replica: e0648550a65b45c3aa9626b9c15e25c9, State: Running, Role: LEADER
I20260810 07:20:00.284451 27150 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0648550a65b45c3aa9626b9c15e25c9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33921 } }
I20260810 07:20:00.284816 27151 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0648550a65b45c3aa9626b9c15e25c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0648550a65b45c3aa9626b9c15e25c9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33921 } } }
I20260810 07:20:00.284835 27152 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0648550a65b45c3aa9626b9c15e25c9. Latest consensus state: current_term: 1 leader_uuid: "e0648550a65b45c3aa9626b9c15e25c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0648550a65b45c3aa9626b9c15e25c9" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 33921 } } }
I20260810 07:20:00.284973 27151 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:00.285073 27152 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:00.285355 27165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:00.285948 27165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:00.289535 27165 catalog_manager.cc:1347] Generated new cluster ID: a8f56bda434142848778757685357d15
I20260810 07:20:00.289598 27165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:00.297660 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.285112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.14.164.62:33921)
0810 07:20:00.285332 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.285343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.285381 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:20:00.285530 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.285540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.285608 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.285766 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.286872 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.288274 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.289241 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.289941 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.290060 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.293397 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.293403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.293403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.293418 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.295793 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.296259 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.296262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.296263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.296383 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.296612 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.296613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.296613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.296962 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.297130 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.297256 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.297419 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260810 07:20:00.298277 27165 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:00.298841 27165 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:00.312099 27165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Generated new TSK 0
I20260810 07:20:00.312356 27165 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab.
Entry for principal oryx/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab.
I20260810 07:20:00.348982 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33921
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:00.439108 27186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:00.439348 27186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:00.439375 27186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:00.440610 27186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:00.440661 27186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:00.440682 27186 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:20:00.440701 27186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:20:00.442399 27186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:33921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:00.442670 27186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:00.442957 27186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:00.448622 27191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.448606 27192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.448606 27195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:00.448684 27186 server_base.cc:1034] running on GCE node
I20260810 07:20:00.449311 27186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:00.449620 27186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:00.450789 27186 hybrid_clock.cc:648] HybridClock initialized: now 1786346400450761 us; error 50 us; skew 500 ppm
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:00.454066 27186 init.cc:377] Logged in from keytab as oryx/127.14.164.1@KRBTEST.COM (short username oryx)
I20260810 07:20:00.454594 27186 webserver.cc:460] Webserver started at http://127.14.164.1:42313/ using document root <none> and password file <none>
I20260810 07:20:00.454908 27186 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:00.454962 27186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:00.455102 27186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:00.456239 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "47c53d9ffc02a5fe9bd501415d0930ca"
server_key_iv: "3e7ce29e1e03b54bf40ad40311826abb"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.456631 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "47c53d9ffc02a5fe9bd501415d0930ca"
server_key_iv: "3e7ce29e1e03b54bf40ad40311826abb"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.458324 27186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.459246 27207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.459532 27186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.459632 27186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "47c53d9ffc02a5fe9bd501415d0930ca"
server_key_iv: "3e7ce29e1e03b54bf40ad40311826abb"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.459733 27186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:00.471885 27186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:00.474625 27186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:00.474839 27186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:00.475452 27186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:00.475533 27186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.475572 27186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:00.475628 27186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.482847 27186 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:44285
I20260810 07:20:00.482915 27393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:44285 every 8 connection(s)
I20260810 07:20:00.483285 27186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:00.485900 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 27186
I20260810 07:20:00.486080 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:00.486382 14992 external_mini_cluster.cc:1428] Setting key 6def17b5d6288fd4b1ff2b6b77231ae0
I20260810 07:20:00.488292 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.476046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37463 (local address 127.14.164.62:33921)
0810 07:20:00.476096 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.476098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.476326 (+ 228us) server_negotiation.cc:407] Connection header received
0810 07:20:00.477005 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.477006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.477020 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.477080 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.478095 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.478953 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.480765 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.481003 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.481078 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.484146 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.484147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.484148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.484156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.486198 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.486771 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.486773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.486773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.486849 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.487221 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.487222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.487222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.487423 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.487529 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.488042 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.488154 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:20:00.488332 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.476003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.14.164.1:37463)
0810 07:20:00.476191 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.476231 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.476890 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.477155 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.477160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.477613 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:00.477997 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.478004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.479070 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.479071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.480635 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.480639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.481101 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.481102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.481282 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.481972 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:00.481984 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:00.483994 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:00.486300 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.486308 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.486310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.486695 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.486940 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.486941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.486942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.487117 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.487552 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:00.487560 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:00.487873 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.487990 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.488150 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260810 07:20:00.489616 27395 heartbeater.cc:347] Connected to a master server at 127.14.164.62:33921
I20260810 07:20:00.489733 27395 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:00.489946 27395 heartbeater.cc:511] Master 127.14.164.62:33921 requested a full tablet report, sending...
I20260810 07:20:00.490537 27071 ts_manager.cc:194] Registered new tserver with Master: a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285)
I20260810 07:20:00.491317 27071 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:37463
I20260810 07:20:00.496673 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.490404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:35019 (local address 127.14.164.62:33921)
0810 07:20:00.490445 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.490447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.490457 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:00.490482 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.490482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.490494 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.490562 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.490965 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.491444 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.492410 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.492638 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.492751 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.493771 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.493773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.493773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.493782 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.495362 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.495814 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.495816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.495816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.495884 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.496090 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.496091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.496091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.496221 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.496300 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.496467 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.496568 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:20:00.496686 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.490206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.14.164.1:35019)
0810 07:20:00.490376 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.490389 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.490439 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.490569 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.490570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.490806 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:00.490875 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.490877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.491549 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.491550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.492321 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.492324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.492744 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.492745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.492826 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.493334 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:00.493340 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:00.493681 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:00.495492 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.495494 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.495494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.495692 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.495953 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.495954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.495955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.496024 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.496304 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:00.496305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:00.496359 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.496440 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.496568 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for oryx/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab.
Entry for principal oryx/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab.
I20260810 07:20:00.521325 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33921
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:00.620754 27408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:00.621116 27408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:00.621161 27408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:00.623389 27408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:00.623481 27408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:00.623526 27408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:20:00.623564 27408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:20:00.626391 27408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:33921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:00.626706 27408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:00.627019 27408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:00.636394 27418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.636379 27420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.636375 27422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:00.636642 27408 server_base.cc:1034] running on GCE node
I20260810 07:20:00.637027 27408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:00.637362 27408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:00.638515 27408 hybrid_clock.cc:648] HybridClock initialized: now 1786346400638490 us; error 41 us; skew 500 ppm
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:00.642513 27408 init.cc:377] Logged in from keytab as oryx/127.14.164.2@KRBTEST.COM (short username oryx)
I20260810 07:20:00.647176 27408 webserver.cc:460] Webserver started at http://127.14.164.2:40745/ using document root <none> and password file <none>
I20260810 07:20:00.647584 27408 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:00.647679 27408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:00.647852 27408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:00.649784 27408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "83b32205b2804570af172f289ea2276b"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "125ed09b06332bdda99ac0e9fee39da6"
server_key_iv: "a235106ea4230ac182c546075ffa5ffa"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.650332 27408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "83b32205b2804570af172f289ea2276b"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "125ed09b06332bdda99ac0e9fee39da6"
server_key_iv: "a235106ea4230ac182c546075ffa5ffa"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.652848 27408 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260810 07:20:00.654002 27430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.654243 27408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.654364 27408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "83b32205b2804570af172f289ea2276b"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "125ed09b06332bdda99ac0e9fee39da6"
server_key_iv: "a235106ea4230ac182c546075ffa5ffa"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.654457 27408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:00.684469 27408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:00.687296 27408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:00.687469 27408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:00.688050 27408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:00.688095 27408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.688125 27408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:00.688174 27408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.695380 27408 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:43609
I20260810 07:20:00.695598 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:43609 every 8 connection(s)
I20260810 07:20:00.695854 27408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:00.700156 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.688688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:47665 (local address 127.14.164.62:33921)
0810 07:20:00.688822 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.688824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.689060 (+ 236us) server_negotiation.cc:407] Connection header received
0810 07:20:00.689692 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.689693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.689718 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.689781 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.690908 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.691853 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.693688 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.694006 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.694094 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.697205 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.697207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.697208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.697216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.698684 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.699129 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.699131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.699131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.699192 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.699471 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.699472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.699472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.699600 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.699670 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.699973 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.700061 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 07:20:00.700353 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.688646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.14.164.2:47665)
0810 07:20:00.688914 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.688956 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.689574 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.689875 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.689879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.690445 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:00.690771 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.690779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.692001 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.692001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.693590 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.693593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.694123 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.694125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.694297 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.695055 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:00.695066 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:00.697088 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:00.698795 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.698802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.698806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.699043 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.699276 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.699277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.699277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.699398 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.699673 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:00.699681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:00.699833 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.699934 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.700106 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":139,"threads_started":1}
I20260810 07:20:00.700981 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 27408
I20260810 07:20:00.701095 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:00.701335 14992 external_mini_cluster.cc:1428] Setting key 3874fab12c1901f783b0eac3d4c9b78c
I20260810 07:20:00.701831 27616 heartbeater.cc:347] Connected to a master server at 127.14.164.62:33921
I20260810 07:20:00.702069 27616 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:00.702329 27616 heartbeater.cc:511] Master 127.14.164.62:33921 requested a full tablet report, sending...
I20260810 07:20:00.702764 27071 ts_manager.cc:194] Registered new tserver with Master: 83b32205b2804570af172f289ea2276b (127.14.164.2:43609)
I20260810 07:20:00.703179 27071 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:47665
I20260810 07:20:00.709805 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.702767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.14.164.2:45229)
0810 07:20:00.703365 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.703412 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.703496 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.703689 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.703690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.703931 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:00.704018 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.704020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.704858 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.704859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.705623 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.705626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.706123 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.706123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.706175 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.706738 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:00.706742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:00.707045 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:00.708617 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.708619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.708619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.708880 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.709097 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.709098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.709098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.709162 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.709453 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:00.709454 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:00.709504 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.709580 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.709691 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":83,"threads_started":1}
I20260810 07:20:00.709832 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.702798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:45229 (local address 127.14.164.62:33921)
0810 07:20:00.702839 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.702842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.703478 (+ 636us) server_negotiation.cc:407] Connection header received
0810 07:20:00.703595 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.703596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.703611 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.703676 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.704131 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.704680 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.705710 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.706043 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.706124 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.707139 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.707140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.707141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.707148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.708533 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.708963 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.708965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.708965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.709025 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.709222 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.709223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.709224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.709375 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.709448 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.709618 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.709730 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab.
Entry for principal oryx/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab.
I20260810 07:20:00.732059 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:33921
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:00.820111 27632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:00.820338 27632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:00.820363 27632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:00.821563 27632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:00.821606 27632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:00.821626 27632 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:20:00.821643 27632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:20:00.823101 27632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.164.62:33921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:00.823298 27632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:00.823482 27632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:00.828800 27641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.828800 27642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:00.828768 27646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:00.829064 27632 server_base.cc:1034] running on GCE node
I20260810 07:20:00.829310 27632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:00.829519 27632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:00.830698 27632 hybrid_clock.cc:648] HybridClock initialized: now 1786346400830687 us; error 38 us; skew 500 ppm
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:00.833386 27632 init.cc:377] Logged in from keytab as oryx/127.14.164.3@KRBTEST.COM (short username oryx)
I20260810 07:20:00.833796 27632 webserver.cc:460] Webserver started at http://127.14.164.3:37443/ using document root <none> and password file <none>
I20260810 07:20:00.833988 27632 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:00.834031 27632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:00.834123 27632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:00.835225 27632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "c101f1ee7a6342289017fefa67104457"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "d8712b23d9ae95bdf57dfb280edd1c6a"
server_key_iv: "d0726cde95218531cbe5ff97f913aae6"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.835556 27632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "c101f1ee7a6342289017fefa67104457"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "d8712b23d9ae95bdf57dfb280edd1c6a"
server_key_iv: "d0726cde95218531cbe5ff97f913aae6"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.837033 27632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:00.837921 27653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.838281 27632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.838410 27632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "c101f1ee7a6342289017fefa67104457"
format_stamp: "Formatted at 2026-08-10 07:20:00 on dist-test-slave-x5fp"
server_key: "d8712b23d9ae95bdf57dfb280edd1c6a"
server_key_iv: "d0726cde95218531cbe5ff97f913aae6"
server_key_version: "encryptionkey@0"
I20260810 07:20:00.838513 27632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:00.855755 27632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:00.858271 27632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:00.858431 27632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:00.859031 27632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:00.859077 27632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.859103 27632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:00.859115 27632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:00.866788 27632 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:37759
I20260810 07:20:00.866861 27838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:37759 every 8 connection(s)
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:00.867194 27632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:00.868227 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 27632
I20260810 07:20:00.868342 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:00.868575 14992 external_mini_cluster.cc:1428] Setting key f25b0109f384bf97df57d10224f73640
I20260810 07:20:00.870334 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.859565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:47197 (local address 127.14.164.62:33921)
0810 07:20:00.859643 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.859646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.859830 (+ 184us) server_negotiation.cc:407] Connection header received
0810 07:20:00.860511 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.860512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.860528 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.860614 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.861655 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.862624 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.864373 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.864693 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.864817 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.867461 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.867463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.867463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.867471 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.869179 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.869621 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.869622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.869623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.869681 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.869861 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.869862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.869862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.869962 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.870008 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.870204 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.870267 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:20:00.870523 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.859517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.14.164.3:47197)
0810 07:20:00.859689 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.859733 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.860379 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.860711 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.860715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.861132 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:00.861501 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.861510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.862753 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.862754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.864255 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.864260 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.864840 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.864841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.865016 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.865651 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:00.865659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:00.867361 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:00.869252 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.869263 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.869265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.869570 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.869727 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.869728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.869728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.869819 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.869997 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:00.870000 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:00.870114 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.870193 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.870322 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260810 07:20:00.871004 27839 heartbeater.cc:347] Connected to a master server at 127.14.164.62:33921
I20260810 07:20:00.871133 27839 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:00.871353 27839 heartbeater.cc:511] Master 127.14.164.62:33921 requested a full tablet report, sending...
I20260810 07:20:00.871869 27071 ts_manager.cc:194] Registered new tserver with Master: c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:00.872404 27071 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:47197
I20260810 07:20:00.878178 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.872067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:48195 (local address 127.14.164.62:33921)
0810 07:20:00.872112 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.872114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.872218 (+ 104us) server_negotiation.cc:407] Connection header received
0810 07:20:00.872248 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.872249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.872260 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.872337 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.872663 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.873393 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.874179 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.874384 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.874458 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.875472 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.875474 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.875474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.875481 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.877002 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.877364 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.877365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.877365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.877421 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.877635 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.877636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.877636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.877761 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.877899 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.878040 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.878108 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:20:00.878207 27848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.872012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.14.164.3:48195)
0810 07:20:00.872150 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.872164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.872211 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.872337 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.872338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.872531 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:00.872599 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.872602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.873525 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.873526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.874084 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.874087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.874478 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.874479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.874540 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.875086 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:00.875091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:00.875387 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:00.877105 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.877106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.877107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.877294 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.877481 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:00.877482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:00.877482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:00.877543 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:00.877893 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:00.877895 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:00.877942 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.878015 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.878122 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260810 07:20:00.880499 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:00.905515 27163 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:00.899221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.14.164.62:33921)
0810 07:20:00.899287 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.899289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.899300 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:20:00.899376 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.899377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.899393 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.899461 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.900093 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.900591 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.901413 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.901668 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.901757 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.905365 (+ 3608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42382: BlockingRecv error: recv got EOF from 127.0.0.1:42382 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 07:20:00 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:00.913755 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.906302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.14.164.62:33921)
0810 07:20:00.906352 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.906354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.906393 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:20:00.906450 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.906451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.906460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.906512 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.907176 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.907664 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.908390 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.908586 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.908651 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.910930 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.910931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.910932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.910938 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.912509 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.912973 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.912974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.912974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.913040 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.913261 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.913262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.913262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.913371 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.913448 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.913615 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.913683 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:20:00.915863 27071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:00.917752 27071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:00.924978 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.920736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:37759 (local address 127.0.0.1:57974)
0810 07:20:00.920963 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.920975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.921036 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.921247 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.921251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.921253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:00.921334 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.921353 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.922733 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.922733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.923869 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.923871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.924575 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.924576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.924667 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.924685 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.924793 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.924865 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":26,"thread_start_us":89,"threads_started":1}
I20260810 07:20:00.925099 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.920694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.14.164.2:43609)
0810 07:20:00.921177 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.921180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.921190 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:00.921301 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.921303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.921320 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.921409 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:00.921608 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.922341 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.923831 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.924562 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.924637 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.924741 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.924944 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.924988 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":125,"threads_started":1}
I20260810 07:20:00.925218 27876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.920482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:43609 (local address 127.0.0.1:43484)
0810 07:20:00.920880 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.920920 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.920991 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.921481 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.921482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.921484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:00.921524 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.921525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.922439 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.922440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.923749 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.923752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.924780 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.924782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.924851 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.924854 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.924924 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.924973 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":84,"threads_started":1}
I20260810 07:20:00.925282 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.920839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.14.164.3:37759)
0810 07:20:00.921028 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.921034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.921046 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:00.921115 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.921117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.921150 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.921237 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:00.921513 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.922651 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.923953 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.924486 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.924573 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.924665 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.925131 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.925187 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260810 07:20:00.925593 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.920627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.14.164.1:44285)
0810 07:20:00.920903 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.920909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.921064 (+ 155us) server_negotiation.cc:407] Connection header received
0810 07:20:00.921194 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.921196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.921212 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.921281 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:00.921529 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.922637 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.923789 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.924562 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.924637 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.924773 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.924962 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.925023 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":120,"threads_started":1}
I20260810 07:20:00.925846 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.920471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44285 (local address 127.0.0.1:33430)
0810 07:20:00.920670 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:00.921009 (+ 339us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:00.921090 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:00.921367 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:00.921368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:00.921370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:00.921434 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.921436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.922745 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.922746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.923696 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:00.923699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.924795 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.924796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:00.924853 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.924859 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:20:00.925732 (+ 873us) client_negotiation.cc:240] Negotiation successful
0810 07:20:00.925778 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260810 07:20:00.926019 27504 tablet_service.cc:1467] Processing CreateTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd (DEFAULT_TABLE table=test-table [id=cdc5af14534e46228d39d5d92b45d16c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.926476 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.926528 27285 tablet_service.cc:1467] Processing CreateTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd (DEFAULT_TABLE table=test-table [id=cdc5af14534e46228d39d5d92b45d16c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.926903 27285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.929246 27728 tablet_service.cc:1467] Processing CreateTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd (DEFAULT_TABLE table=test-table [id=cdc5af14534e46228d39d5d92b45d16c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.929642 27728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.930168 27887 tablet_bootstrap.cc:492] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Bootstrap starting.
I20260810 07:20:00.930477 27888 tablet_bootstrap.cc:492] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Bootstrap starting.
I20260810 07:20:00.930953 27887 tablet_bootstrap.cc:654] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.931125 27888 tablet_bootstrap.cc:654] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.931329 27887 log.cc:824] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:00.931461 27888 log.cc:824] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:00.932137 27887 tablet_bootstrap.cc:492] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: No bootstrap required, opened a new log
I20260810 07:20:00.932222 27888 tablet_bootstrap.cc:492] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: No bootstrap required, opened a new log
I20260810 07:20:00.932225 27887 ts_tablet_manager.cc:1397] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:00.932289 27888 ts_tablet_manager.cc:1397] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:00.933331 27891 tablet_bootstrap.cc:492] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Bootstrap starting.
I20260810 07:20:00.934054 27891 tablet_bootstrap.cc:654] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.934185 27887 raft_consensus.cc:348] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:00.934350 27887 raft_consensus.cc:374] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.934394 27887 raft_consensus.cc:729] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b32205b2804570af172f289ea2276b, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.934286 27888 raft_consensus.cc:348] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:00.934399 27891 log.cc:824] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:00.934454 27888 raft_consensus.cc:374] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.934499 27888 raft_consensus.cc:729] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7fd8b48ad4ccc9d34232d0a66838f, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.934513 27887 consensus_queue.cc:260] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:00.934633 27888 consensus_queue.cc:260] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:00.934795 27887 ts_tablet_manager.cc:1428] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:00.934880 27616 heartbeater.cc:503] Master 127.14.164.62:33921 was elected leader, sending a full tablet report...
I20260810 07:20:00.934919 27888 ts_tablet_manager.cc:1428] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:00.935038 27395 heartbeater.cc:503] Master 127.14.164.62:33921 was elected leader, sending a full tablet report...
I20260810 07:20:00.936102 27891 tablet_bootstrap.cc:492] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: No bootstrap required, opened a new log
I20260810 07:20:00.936178 27891 ts_tablet_manager.cc:1397] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:20:00.937674 27891 raft_consensus.cc:348] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:00.937830 27891 raft_consensus.cc:374] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.937888 27891 raft_consensus.cc:729] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c101f1ee7a6342289017fefa67104457, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.938010 27891 consensus_queue.cc:260] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:00.938270 27891 ts_tablet_manager.cc:1428] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 07:20:00.938319 27839 heartbeater.cc:503] Master 127.14.164.62:33921 was elected leader, sending a full tablet report...
W20260810 07:20:00.946494 27617 tablet.cc:2369] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:20:00.984241 27396 tablet.cc:2369] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:01.108919 27893 raft_consensus.cc:482] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:01.109076 27893 raft_consensus.cc:504] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.109539 27893 leader_election.cc:288] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:01.113927 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.109780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:37759 (local address 127.14.164.2:54611)
0810 07:20:01.109868 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.109883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.109943 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.110128 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.110129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.110131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:01.110213 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.110215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.111202 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.111203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.112661 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.112664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.113573 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.113574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.113672 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.113680 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.113755 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.113816 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":93}
I20260810 07:20:01.113983 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.109782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44285 (local address 127.14.164.2:53585)
0810 07:20:01.110135 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.110146 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.110209 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.110395 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.110396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.110397 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:01.110448 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.110449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.111276 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.111277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.112660 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.112664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.113573 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.113574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.113672 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.113680 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.113755 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.113816 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":137,"thread_start_us":146,"threads_started":1}
I20260810 07:20:01.113969 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.109869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:53585 (local address 127.14.164.1:44285)
0810 07:20:01.109920 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.109922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.110216 (+ 294us) server_negotiation.cc:407] Connection header received
0810 07:20:01.110287 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.110289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.110299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.110379 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:01.110518 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.111189 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.112812 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.113420 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.113522 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.113532 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.113821 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.113881 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:20:01.113970 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.109885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:54611 (local address 127.14.164.3:37759)
0810 07:20:01.109932 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.109934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.109946 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:01.110014 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.110015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.110026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.110095 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:01.110299 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.111078 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.112869 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.113443 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.113521 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.113532 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.113812 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.113868 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:20:01.114423 27759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cb1af0a4c9492a85d3ba7b7b1d4fcd" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c101f1ee7a6342289017fefa67104457" is_pre_election: true
I20260810 07:20:01.114418 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cb1af0a4c9492a85d3ba7b7b1d4fcd" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" is_pre_election: true
I20260810 07:20:01.114627 27759 raft_consensus.cc:2454] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 0.
I20260810 07:20:01.114629 27314 raft_consensus.cc:2454] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 0.
I20260810 07:20:01.114948 27431 leader_election.cc:302] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, a5e7fd8b48ad4ccc9d34232d0a66838f; no voters:
I20260810 07:20:01.115118 27893 raft_consensus.cc:2781] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:01.115188 27893 raft_consensus.cc:482] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:01.115207 27893 raft_consensus.cc:3037] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.116283 27893 raft_consensus.cc:504] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.116461 27893 leader_election.cc:288] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 election: Requested vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:01.116672 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cb1af0a4c9492a85d3ba7b7b1d4fcd" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
I20260810 07:20:01.116672 27759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cb1af0a4c9492a85d3ba7b7b1d4fcd" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c101f1ee7a6342289017fefa67104457"
I20260810 07:20:01.116830 27759 raft_consensus.cc:3037] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.116842 27314 raft_consensus.cc:3037] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.117678 27759 raft_consensus.cc:2454] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 1.
I20260810 07:20:01.117877 27314 raft_consensus.cc:2454] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 1.
I20260810 07:20:01.117903 27438 leader_election.cc:302] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.118045 27893 raft_consensus.cc:2781] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Leader election won for term 1
W20260810 07:20:01.118104 27841 tablet.cc:2369] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:01.118212 27893 raft_consensus.cc:686] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 1 LEADER]: Becoming Leader. State: Replica: 83b32205b2804570af172f289ea2276b, State: Running, Role: LEADER
I20260810 07:20:01.118305 27893 consensus_queue.cc:237] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.119500 27070 catalog_manager.cc:5495] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b reported cstate change: term changed from 0 to 1, leader changed from <none> to 83b32205b2804570af172f289ea2276b (127.14.164.2). New cstate: current_term: 1 leader_uuid: "83b32205b2804570af172f289ea2276b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:01.156538 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.153872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.14.164.2:43609)
0810 07:20:01.153930 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.153932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.153946 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:01.153989 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.153989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.154010 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.154094 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:01.154243 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.154721 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.155578 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.155774 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.155851 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.156089 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.156206 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.156284 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.156405 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.156435 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:20:01.159318 27759 raft_consensus.cc:1261] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Refusing update from remote peer 83b32205b2804570af172f289ea2276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.159756 27314 raft_consensus.cc:1261] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Refusing update from remote peer 83b32205b2804570af172f289ea2276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.159811 27893 consensus_queue.cc:1035] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.160215 27898 consensus_queue.cc:1035] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.163751 27905 mvcc.cc:205] Tried to move back new op lower bound from 7316874859144609792 to 7316874858981171200. Current Snapshot: MvccSnapshot[applied={T|T < 7316874859144609792}]
I20260810 07:20:01.165871 27908 mvcc.cc:205] Tried to move back new op lower bound from 7316874859144609792 to 7316874858981171200. Current Snapshot: MvccSnapshot[applied={T|T < 7316874859144609792}]
I20260810 07:20:01.166412 27070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:01.166532 27070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:01.167384 27070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Sending DeleteTablet for 3 replicas of tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd
I20260810 07:20:01.167740 27502 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:43484
I20260810 07:20:01.167923 27728 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57974
I20260810 07:20:01.168056 27285 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:33430
I20260810 07:20:01.168040 27071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:01.168206 27071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:01.167747 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:43484
I20260810 07:20:01.168853 27922 tablet_replica.cc:330] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: stopping tablet replica
I20260810 07:20:01.168987 27922 raft_consensus.cc:2229] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:01.169075 27922 raft_consensus.cc:2258] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.168639 27923 tablet_replica.cc:330] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: stopping tablet replica
I20260810 07:20:01.169574 27923 raft_consensus.cc:2229] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:01.169596 27285 tablet_service.cc:1467] Processing CreateTablet for tablet fbf47250aa544f80b6f33beb4c5a25e1 (DEFAULT_TABLE table=test-table [id=2d13307ebeb04885b341faae2b76d4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.169677 27923 raft_consensus.cc:2258] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.169713 27285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbf47250aa544f80b6f33beb4c5a25e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.169664 27504 tablet_service.cc:1467] Processing CreateTablet for tablet fbf47250aa544f80b6f33beb4c5a25e1 (DEFAULT_TABLE table=test-table [id=2d13307ebeb04885b341faae2b76d4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.169664 27728 tablet_service.cc:1467] Processing CreateTablet for tablet fbf47250aa544f80b6f33beb4c5a25e1 (DEFAULT_TABLE table=test-table [id=2d13307ebeb04885b341faae2b76d4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.169931 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbf47250aa544f80b6f33beb4c5a25e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.169953 27728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbf47250aa544f80b6f33beb4c5a25e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.170166 27923 ts_tablet_manager.cc:1905] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.171240 27888 tablet_bootstrap.cc:492] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Bootstrap starting.
I20260810 07:20:01.171485 27891 tablet_bootstrap.cc:492] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Bootstrap starting.
I20260810 07:20:01.171487 27887 tablet_bootstrap.cc:492] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Bootstrap starting.
I20260810 07:20:01.171669 27888 tablet_bootstrap.cc:654] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.171913 27887 tablet_bootstrap.cc:654] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.171936 27891 tablet_bootstrap.cc:654] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.172137 27923 ts_tablet_manager.cc:1918] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:01.172204 27923 log.cc:1196] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/f3cb1af0a4c9492a85d3ba7b7b1d4fcd
I20260810 07:20:01.172478 27920 tablet_replica.cc:330] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: stopping tablet replica
I20260810 07:20:01.172498 27923 ts_tablet_manager.cc:1939] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting consensus metadata
I20260810 07:20:01.172581 27920 raft_consensus.cc:2229] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:01.172706 27920 raft_consensus.cc:2258] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.173272 27922 ts_tablet_manager.cc:1905] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.173260 27045 catalog_manager.cc:4841] TS a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285): tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd (table test-table [id=cdc5af14534e46228d39d5d92b45d16c]) successfully deleted
I20260810 07:20:01.173319 27891 tablet_bootstrap.cc:492] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: No bootstrap required, opened a new log
I20260810 07:20:01.173370 27891 ts_tablet_manager.cc:1397] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:20:01.173527 27891 raft_consensus.cc:348] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.173609 27891 raft_consensus.cc:374] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.173629 27891 raft_consensus.cc:729] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c101f1ee7a6342289017fefa67104457, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.173684 27891 consensus_queue.cc:260] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.173779 27891 ts_tablet_manager.cc:1428] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.174541 27887 tablet_bootstrap.cc:492] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: No bootstrap required, opened a new log
I20260810 07:20:01.174557 27922 ts_tablet_manager.cc:1918] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:01.174593 27887 ts_tablet_manager.cc:1397] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:20:01.174611 27922 log.cc:1196] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/f3cb1af0a4c9492a85d3ba7b7b1d4fcd
I20260810 07:20:01.174736 27887 raft_consensus.cc:348] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.174799 27922 ts_tablet_manager.cc:1939] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P c101f1ee7a6342289017fefa67104457: Deleting consensus metadata
I20260810 07:20:01.174813 27887 raft_consensus.cc:374] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.174834 27887 raft_consensus.cc:729] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b32205b2804570af172f289ea2276b, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.174875 27887 consensus_queue.cc:260] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.175004 27887 ts_tablet_manager.cc:1428] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.175678 27049 catalog_manager.cc:4841] TS c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759): tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd (table test-table [id=cdc5af14534e46228d39d5d92b45d16c]) successfully deleted
I20260810 07:20:01.175841 27920 ts_tablet_manager.cc:1905] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.177122 27920 ts_tablet_manager.cc:1918] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:01.177181 27920 log.cc:1196] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/f3cb1af0a4c9492a85d3ba7b7b1d4fcd
I20260810 07:20:01.177380 27920 ts_tablet_manager.cc:1939] T f3cb1af0a4c9492a85d3ba7b7b1d4fcd P 83b32205b2804570af172f289ea2276b: Deleting consensus metadata
I20260810 07:20:01.177908 27049 catalog_manager.cc:4841] TS 83b32205b2804570af172f289ea2276b (127.14.164.2:43609): tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd (table test-table [id=cdc5af14534e46228d39d5d92b45d16c]) successfully deleted
W20260810 07:20:01.178125 27049 catalog_manager.cc:4808] TS 83b32205b2804570af172f289ea2276b (127.14.164.2:43609): delete failed for tablet f3cb1af0a4c9492a85d3ba7b7b1d4fcd because the tablet was not found. No further retry: Not found: Tablet not found: f3cb1af0a4c9492a85d3ba7b7b1d4fcd
I20260810 07:20:01.181368 27888 tablet_bootstrap.cc:492] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: No bootstrap required, opened a new log
I20260810 07:20:01.181469 27888 ts_tablet_manager.cc:1397] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260810 07:20:01.181813 27888 raft_consensus.cc:348] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.181921 27888 raft_consensus.cc:374] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.181953 27888 raft_consensus.cc:729] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7fd8b48ad4ccc9d34232d0a66838f, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.182006 27888 consensus_queue.cc:260] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.182116 27888 ts_tablet_manager.cc:1428] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:01.258033 27896 raft_consensus.cc:482] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:01.258159 27896 raft_consensus.cc:504] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.258543 27896 leader_election.cc:288] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), 83b32205b2804570af172f289ea2276b (127.14.164.2:43609)
I20260810 07:20:01.262912 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.258870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:41177 (local address 127.14.164.1:44285)
0810 07:20:01.258927 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.258929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.259270 (+ 341us) server_negotiation.cc:407] Connection header received
0810 07:20:01.259420 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.259421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.259434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.259487 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:01.259705 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.260451 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.261951 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.262491 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.262545 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.262555 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.262784 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.262830 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:20:01.263190 27930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.258766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44285 (local address 127.14.164.3:41177)
0810 07:20:01.259183 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.259204 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.259302 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.259534 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.259535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.259537 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:01.259624 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.259626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.260557 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.260558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.261832 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.261836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.262622 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.262623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.262700 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.262704 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.263027 (+ 323us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.263080 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":248,"thread_start_us":109,"threads_started":1}
I20260810 07:20:01.263496 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbf47250aa544f80b6f33beb4c5a25e1" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" is_pre_election: true
I20260810 07:20:01.263574 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.258765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:43609 (local address 127.14.164.3:53203)
0810 07:20:01.258872 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.258884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.258932 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.259179 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.259180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.259181 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:01.259283 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.259285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.260472 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.260473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.261959 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.261963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.263232 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.263233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.263346 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.263350 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.263412 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.263469 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":1}
I20260810 07:20:01.263598 27314 raft_consensus.cc:2454] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 0.
I20260810 07:20:01.263782 27654 leader_election.cc:302] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e7fd8b48ad4ccc9d34232d0a66838f, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.263916 27896 raft_consensus.cc:2781] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:01.263952 27896 raft_consensus.cc:482] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:01.263973 27896 raft_consensus.cc:3037] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.264371 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.258971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:53203 (local address 127.14.164.2:43609)
0810 07:20:01.259032 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.259034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.259042 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:20:01.259072 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.259073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.259085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.259133 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:01.259360 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.260329 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.262125 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.262844 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.262925 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.262935 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.264214 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.264276 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:20:01.264628 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbf47250aa544f80b6f33beb4c5a25e1" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b32205b2804570af172f289ea2276b" is_pre_election: true
I20260810 07:20:01.264820 27534 raft_consensus.cc:2454] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 0.
I20260810 07:20:01.264968 27896 raft_consensus.cc:504] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.265123 27896 leader_election.cc:288] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 election: Requested vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), 83b32205b2804570af172f289ea2276b (127.14.164.2:43609)
I20260810 07:20:01.265272 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbf47250aa544f80b6f33beb4c5a25e1" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b32205b2804570af172f289ea2276b"
I20260810 07:20:01.265355 27534 raft_consensus.cc:3037] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.265349 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbf47250aa544f80b6f33beb4c5a25e1" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
I20260810 07:20:01.265426 27314 raft_consensus.cc:3037] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.266259 27314 raft_consensus.cc:2454] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 1.
I20260810 07:20:01.266383 27534 raft_consensus.cc:2454] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 1.
I20260810 07:20:01.266439 27654 leader_election.cc:302] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e7fd8b48ad4ccc9d34232d0a66838f, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.266523 27896 raft_consensus.cc:2781] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:01.266625 27896 raft_consensus.cc:686] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 1 LEADER]: Becoming Leader. State: Replica: c101f1ee7a6342289017fefa67104457, State: Running, Role: LEADER
I20260810 07:20:01.266685 27896 consensus_queue.cc:237] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.267522 27070 catalog_manager.cc:5495] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 reported cstate change: term changed from 0 to 1, leader changed from <none> to c101f1ee7a6342289017fefa67104457 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } health_report { overall_health: HEALTHY } } }
Aug 10 07:20:01 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.62@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:01.317574 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.308000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.14.164.62:33921)
0810 07:20:01.308065 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.308074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.308296 (+ 222us) server_negotiation.cc:407] Connection header received
0810 07:20:01.308374 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.308375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.308390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.308456 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:01.309117 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.309894 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.310938 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.311234 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.311307 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.314021 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:01.314022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:01.314022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:01.314031 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:01.315736 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:01.316566 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:01.316568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:01.316568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:01.316645 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:01.316902 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:01.316903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:01.316904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:01.317050 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:01.317128 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.317293 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.317475 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":33}
I20260810 07:20:01.317574 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.307939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:33921 (local address 127.0.0.1:42412)
0810 07:20:01.308195 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.308208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.308296 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.308464 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.308465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.308948 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:01.309020 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.309024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.310005 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.310007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.310839 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.310842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.311382 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.311383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.311443 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.312185 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:01.312191 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:01.313923 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:01.315866 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:01.315878 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:01.315881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:01.316448 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:01.316760 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:01.316761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:01.316762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:01.316835 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:01.317117 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:01.317126 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:01.317179 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.317253 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.317416 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":150,"threads_started":1}
I20260810 07:20:01.320382 27070 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:42412:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:20:01.320636 27070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:01.322254 27504 tablet_service.cc:1467] Processing CreateTablet for tablet 89d7e6b725814a318232f5f337938d6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=59a2165978d947d1b90a83161b709fb3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:20:01.322276 27728 tablet_service.cc:1467] Processing CreateTablet for tablet 89d7e6b725814a318232f5f337938d6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=59a2165978d947d1b90a83161b709fb3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:20:01.322310 27285 tablet_service.cc:1467] Processing CreateTablet for tablet 89d7e6b725814a318232f5f337938d6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=59a2165978d947d1b90a83161b709fb3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:20:01.322436 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d7e6b725814a318232f5f337938d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.322460 27728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d7e6b725814a318232f5f337938d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.322472 27285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d7e6b725814a318232f5f337938d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.324106 27888 tablet_bootstrap.cc:492] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: Bootstrap starting.
I20260810 07:20:01.324126 27891 tablet_bootstrap.cc:492] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: Bootstrap starting.
I20260810 07:20:01.324501 27888 tablet_bootstrap.cc:654] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.324539 27891 tablet_bootstrap.cc:654] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.324975 27887 tablet_bootstrap.cc:492] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: Bootstrap starting.
I20260810 07:20:01.325304 27887 tablet_bootstrap.cc:654] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.325909 27887 tablet_bootstrap.cc:492] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: No bootstrap required, opened a new log
I20260810 07:20:01.325966 27887 ts_tablet_manager.cc:1397] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:01.326072 27887 raft_consensus.cc:348] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.326162 27887 raft_consensus.cc:374] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.326181 27887 raft_consensus.cc:729] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b32205b2804570af172f289ea2276b, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.326229 27887 consensus_queue.cc:260] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.326316 27887 ts_tablet_manager.cc:1428] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.326360 27898 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.326468 27898 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.326534 27891 tablet_bootstrap.cc:492] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: No bootstrap required, opened a new log
I20260810 07:20:01.326586 27891 ts_tablet_manager.cc:1397] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:20:01.326957 27891 raft_consensus.cc:348] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.327046 27891 raft_consensus.cc:374] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.327066 27891 raft_consensus.cc:729] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c101f1ee7a6342289017fefa67104457, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.327109 27891 consensus_queue.cc:260] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.327200 27891 ts_tablet_manager.cc:1428] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:01.327316 27896 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.327394 27896 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.327921 27888 tablet_bootstrap.cc:492] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: No bootstrap required, opened a new log
I20260810 07:20:01.327976 27888 ts_tablet_manager.cc:1397] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 07:20:01.328138 27888 raft_consensus.cc:348] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.328202 27888 raft_consensus.cc:374] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.328219 27888 raft_consensus.cc:729] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7fd8b48ad4ccc9d34232d0a66838f, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.328272 27888 consensus_queue.cc:260] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.328382 27888 ts_tablet_manager.cc:1428] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.328438 27895 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.328526 27895 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.525054 27896 raft_consensus.cc:482] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:01.525204 27896 raft_consensus.cc:504] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.525457 27896 leader_election.cc:288] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), 83b32205b2804570af172f289ea2276b (127.14.164.2:43609)
I20260810 07:20:01.525774 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d7e6b725814a318232f5f337938d6f" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b32205b2804570af172f289ea2276b" is_pre_election: true
I20260810 07:20:01.525830 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d7e6b725814a318232f5f337938d6f" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" is_pre_election: true
I20260810 07:20:01.525902 27534 raft_consensus.cc:2454] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 0.
I20260810 07:20:01.525956 27314 raft_consensus.cc:2454] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 0.
I20260810 07:20:01.526089 27656 leader_election.cc:302] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.526228 27896 raft_consensus.cc:2781] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:01.526283 27896 raft_consensus.cc:482] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:01.526304 27896 raft_consensus.cc:3037] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.527094 27896 raft_consensus.cc:504] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.527240 27896 leader_election.cc:288] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 election: Requested vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), 83b32205b2804570af172f289ea2276b (127.14.164.2:43609)
I20260810 07:20:01.527462 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d7e6b725814a318232f5f337938d6f" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b32205b2804570af172f289ea2276b"
I20260810 07:20:01.527549 27534 raft_consensus.cc:3037] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.527511 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d7e6b725814a318232f5f337938d6f" candidate_uuid: "c101f1ee7a6342289017fefa67104457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
I20260810 07:20:01.527643 27314 raft_consensus.cc:3037] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.528225 27534 raft_consensus.cc:2454] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 1.
I20260810 07:20:01.528410 27656 leader_election.cc:302] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.528424 27314 raft_consensus.cc:2454] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c101f1ee7a6342289017fefa67104457 in term 1.
I20260810 07:20:01.528517 27896 raft_consensus.cc:2781] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:01.528576 27896 raft_consensus.cc:686] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [term 1 LEADER]: Becoming Leader. State: Replica: c101f1ee7a6342289017fefa67104457, State: Running, Role: LEADER
I20260810 07:20:01.528640 27896 consensus_queue.cc:237] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.529023 27931 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: TxnStatusTablet state changed. Reason: New leader c101f1ee7a6342289017fefa67104457. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.529134 27931 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:20:01.529320 27954 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:20:01.529462 27954 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:20:01.529376 27070 catalog_manager.cc:5495] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 reported cstate change: term changed from 0 to 1, leader changed from <none> to c101f1ee7a6342289017fefa67104457 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } health_report { overall_health: HEALTHY } } }
I20260810 07:20:01.559427 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.556658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.14.164.3:37759)
0810 07:20:01.556794 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.556796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.556806 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:01.556871 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.556872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.556892 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.556958 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:01.557110 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.557633 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.558447 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.558669 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.558764 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.558983 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.559108 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.559192 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.559333 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.559357 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:20:01.559492 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.556558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:37759 (local address 127.0.0.1:57982)
0810 07:20:01.556639 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.556657 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.556740 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.556979 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.556980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.556982 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:01.557043 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.557045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.557751 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.557753 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.558353 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.558356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.558763 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.558764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.558831 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.558851 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.559225 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.559241 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.559331 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.559399 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:20:01.561352 27314 raft_consensus.cc:1261] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Refusing update from remote peer c101f1ee7a6342289017fefa67104457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.561358 27534 raft_consensus.cc:1261] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Refusing update from remote peer c101f1ee7a6342289017fefa67104457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.561743 27931 consensus_queue.cc:1035] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.562032 27931 consensus_queue.cc:1035] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.562750 27895 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: TxnStatusTablet state changed. Reason: New leader c101f1ee7a6342289017fefa67104457. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.562875 27895 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.562975 27898 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: TxnStatusTablet state changed. Reason: New leader c101f1ee7a6342289017fefa67104457. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.563060 27898 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.564028 27931 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.564119 27931 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:20:01.564587 27896 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.564662 27896 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P c101f1ee7a6342289017fefa67104457: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:20:01.564760 27898 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.564869 27898 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P 83b32205b2804570af172f289ea2276b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.565156 27895 tablet_replica.cc:439] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c101f1ee7a6342289017fefa67104457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } } }
I20260810 07:20:01.565264 27895 tablet_replica.cc:442] T 89d7e6b725814a318232f5f337938d6f P a5e7fd8b48ad4ccc9d34232d0a66838f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:01.565343 27906 mvcc.cc:205] Tried to move back new op lower bound from 7316874860791205888 to 7316874860661952512. Current Snapshot: MvccSnapshot[applied={T|T < 7316874860791205888 or (T in {7316874860791205888})}]
I20260810 07:20:01.569301 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.566584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.14.164.3:37759)
0810 07:20:01.566626 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.566629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.566638 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:20:01.566661 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.566662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.566693 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.566745 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:01.566891 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.567398 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.568538 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.568805 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.568883 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.569045 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.569076 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.569114 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.569178 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.569200 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:20:01.575322 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.572347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:36961 (local address 127.14.164.3:37759)
0810 07:20:01.572398 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.572400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.572631 (+ 231us) server_negotiation.cc:407] Connection header received
0810 07:20:01.572657 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.572657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.572667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.572788 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:01.572984 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.573460 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.574589 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.574772 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.574846 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.575044 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.575071 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.575118 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.575216 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.575232 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:20:01.575642 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.572268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:37759 (local address 127.14.164.3:36961)
0810 07:20:01.572511 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:01.572523 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:01.572589 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:01.572796 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:01.572797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:01.572798 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:01.572896 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.572898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.573543 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.573544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.574487 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:01.574490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.574866 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.574867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:01.574938 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.574959 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.575142 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.575145 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:01.575214 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:20:01.575259 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":115,"threads_started":1}
I20260810 07:20:01.578289 27314 raft_consensus.cc:1261] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Refusing update from remote peer c101f1ee7a6342289017fefa67104457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.578483 27534 raft_consensus.cc:1261] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Refusing update from remote peer c101f1ee7a6342289017fefa67104457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.578578 27966 consensus_queue.cc:1035] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.578694 27931 consensus_queue.cc:1035] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.592650 27071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:01.592815 27071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:01.593538 27071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Sending DeleteTablet for 3 replicas of tablet fbf47250aa544f80b6f33beb4c5a25e1
I20260810 07:20:01.593829 27285 tablet_service.cc:1514] Processing DeleteTablet for tablet fbf47250aa544f80b6f33beb4c5a25e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:33430
I20260810 07:20:01.593909 27728 tablet_service.cc:1514] Processing DeleteTablet for tablet fbf47250aa544f80b6f33beb4c5a25e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57974
I20260810 07:20:01.593955 27923 tablet_replica.cc:330] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: stopping tablet replica
I20260810 07:20:01.593962 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet fbf47250aa544f80b6f33beb4c5a25e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:43484
I20260810 07:20:01.594023 27923 raft_consensus.cc:2229] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:01.594059 27922 tablet_replica.cc:330] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: stopping tablet replica
I20260810 07:20:01.594072 27923 raft_consensus.cc:2258] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.594121 27922 raft_consensus.cc:2229] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:01.594264 27922 raft_consensus.cc:2258] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.594367 27920 tablet_replica.cc:330] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: stopping tablet replica
I20260810 07:20:01.594439 27920 raft_consensus.cc:2229] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:01.594501 27920 raft_consensus.cc:2258] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.594647 27922 ts_tablet_manager.cc:1905] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.595017 27920 ts_tablet_manager.cc:1905] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.596640 27922 ts_tablet_manager.cc:1918] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:01.596709 27922 log.cc:1196] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/fbf47250aa544f80b6f33beb4c5a25e1
I20260810 07:20:01.597023 27920 ts_tablet_manager.cc:1918] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:01.597062 27922 ts_tablet_manager.cc:1939] T fbf47250aa544f80b6f33beb4c5a25e1 P c101f1ee7a6342289017fefa67104457: Deleting consensus metadata
I20260810 07:20:01.597091 27920 log.cc:1196] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/fbf47250aa544f80b6f33beb4c5a25e1
I20260810 07:20:01.597324 27920 ts_tablet_manager.cc:1939] T fbf47250aa544f80b6f33beb4c5a25e1 P 83b32205b2804570af172f289ea2276b: Deleting consensus metadata
I20260810 07:20:01.597568 27049 catalog_manager.cc:4841] TS c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759): tablet fbf47250aa544f80b6f33beb4c5a25e1 (table test-table [id=2d13307ebeb04885b341faae2b76d4b0]) successfully deleted
I20260810 07:20:01.597666 27049 catalog_manager.cc:4841] TS 83b32205b2804570af172f289ea2276b (127.14.164.2:43609): tablet fbf47250aa544f80b6f33beb4c5a25e1 (table test-table [id=2d13307ebeb04885b341faae2b76d4b0]) successfully deleted
I20260810 07:20:01.600997 27923 ts_tablet_manager.cc:1905] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.602770 27923 ts_tablet_manager.cc:1918] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:01.602847 27923 log.cc:1196] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/fbf47250aa544f80b6f33beb4c5a25e1
I20260810 07:20:01.603063 27923 ts_tablet_manager.cc:1939] T fbf47250aa544f80b6f33beb4c5a25e1 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting consensus metadata
I20260810 07:20:01.603652 27045 catalog_manager.cc:4841] TS a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285): tablet fbf47250aa544f80b6f33beb4c5a25e1 (table test-table [id=2d13307ebeb04885b341faae2b76d4b0]) successfully deleted
Aug 10 07:20:01 dist-test-slave-x5fp krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:20:01 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:01 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:01.626252 27163 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:01.619529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.14.164.62:33921)
0810 07:20:01.619608 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.619610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.619751 (+ 141us) server_negotiation.cc:407] Connection header received
0810 07:20:01.619837 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.619838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.619849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.619902 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:01.620747 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.621306 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.622399 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.622604 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.622676 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.626170 (+ 3494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42420: BlockingRecv error: recv got EOF from 127.0.0.1:42420 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 07:20:01 dist-test-slave-x5fp krb5kdc[27003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM
I20260810 07:20:01.636130 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.627324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.14.164.62:33921)
0810 07:20:01.627394 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.627397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.627508 (+ 111us) server_negotiation.cc:407] Connection header received
0810 07:20:01.627582 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.627582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.627593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.627651 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:01.628438 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.629219 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.629960 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.630158 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.630254 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.633070 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:01.633072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:01.633072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:01.633080 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:01.634745 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:01.635225 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:01.635226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:01.635227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:01.635315 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:01.635529 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:01.635530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:01.635530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:01.635660 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:01.635766 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.635912 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.636012 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:20:01.637867 27071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:01.638032 27071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:01.639621 27285 tablet_service.cc:1467] Processing CreateTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 (DEFAULT_TABLE table=test-table [id=9399002f7fd74cbeabc40d42c62a9d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.639678 27504 tablet_service.cc:1467] Processing CreateTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 (DEFAULT_TABLE table=test-table [id=9399002f7fd74cbeabc40d42c62a9d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.639707 27728 tablet_service.cc:1467] Processing CreateTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 (DEFAULT_TABLE table=test-table [id=9399002f7fd74cbeabc40d42c62a9d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.639799 27285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.639822 27728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.639842 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.641304 27888 tablet_bootstrap.cc:492] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Bootstrap starting.
I20260810 07:20:01.641307 27891 tablet_bootstrap.cc:492] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Bootstrap starting.
I20260810 07:20:01.641413 27887 tablet_bootstrap.cc:492] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Bootstrap starting.
I20260810 07:20:01.641772 27891 tablet_bootstrap.cc:654] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.641781 27887 tablet_bootstrap.cc:654] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.641772 27888 tablet_bootstrap.cc:654] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.642690 27891 tablet_bootstrap.cc:492] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: No bootstrap required, opened a new log
I20260810 07:20:01.642740 27891 ts_tablet_manager.cc:1397] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:01.642897 27891 raft_consensus.cc:348] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.642957 27891 raft_consensus.cc:374] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.642970 27891 raft_consensus.cc:729] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c101f1ee7a6342289017fefa67104457, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.642997 27887 tablet_bootstrap.cc:492] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: No bootstrap required, opened a new log
I20260810 07:20:01.643045 27887 ts_tablet_manager.cc:1397] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:01.643030 27891 consensus_queue.cc:260] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.643127 27891 ts_tablet_manager.cc:1428] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 07:20:01.643175 27887 raft_consensus.cc:348] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.643245 27887 raft_consensus.cc:374] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.643265 27887 raft_consensus.cc:729] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b32205b2804570af172f289ea2276b, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.643307 27887 consensus_queue.cc:260] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.643405 27887 ts_tablet_manager.cc:1428] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.643859 27888 tablet_bootstrap.cc:492] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: No bootstrap required, opened a new log
I20260810 07:20:01.643904 27888 ts_tablet_manager.cc:1397] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:20:01.644028 27888 raft_consensus.cc:348] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.644089 27888 raft_consensus.cc:374] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.644104 27888 raft_consensus.cc:729] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7fd8b48ad4ccc9d34232d0a66838f, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.644161 27888 consensus_queue.cc:260] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.644255 27888 ts_tablet_manager.cc:1428] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.721084 27898 raft_consensus.cc:482] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:01.721257 27898 raft_consensus.cc:504] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.721522 27898 leader_election.cc:288] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:01.721794 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a2e5ef6f414c04a0c7ef36ae8d1b68" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" is_pre_election: true
I20260810 07:20:01.721793 27759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a2e5ef6f414c04a0c7ef36ae8d1b68" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c101f1ee7a6342289017fefa67104457" is_pre_election: true
I20260810 07:20:01.721933 27759 raft_consensus.cc:2454] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 0.
I20260810 07:20:01.721948 27314 raft_consensus.cc:2454] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 0.
I20260810 07:20:01.722107 27438 leader_election.cc:302] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.722223 27898 raft_consensus.cc:2781] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:01.722278 27898 raft_consensus.cc:482] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:01.722314 27898 raft_consensus.cc:3037] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.723212 27898 raft_consensus.cc:504] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.723407 27898 leader_election.cc:288] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 election: Requested vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:01.723603 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a2e5ef6f414c04a0c7ef36ae8d1b68" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
I20260810 07:20:01.723711 27314 raft_consensus.cc:3037] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.723656 27759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a2e5ef6f414c04a0c7ef36ae8d1b68" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c101f1ee7a6342289017fefa67104457"
I20260810 07:20:01.723763 27759 raft_consensus.cc:3037] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.724514 27759 raft_consensus.cc:2454] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 1.
I20260810 07:20:01.724573 27314 raft_consensus.cc:2454] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 1.
I20260810 07:20:01.724781 27438 leader_election.cc:302] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.724941 27898 raft_consensus.cc:2781] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:01.725080 27898 raft_consensus.cc:686] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 1 LEADER]: Becoming Leader. State: Replica: 83b32205b2804570af172f289ea2276b, State: Running, Role: LEADER
I20260810 07:20:01.725136 27898 consensus_queue.cc:237] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.725828 27071 catalog_manager.cc:5495] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b reported cstate change: term changed from 0 to 1, leader changed from <none> to 83b32205b2804570af172f289ea2276b (127.14.164.2). New cstate: current_term: 1 leader_uuid: "83b32205b2804570af172f289ea2276b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:01.738996 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:01.736614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.14.164.2:43609)
0810 07:20:01.736681 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:01.736683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:01.736692 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:20:01.736743 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:01.736744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:01.736756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:01.736808 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:01.736991 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.737494 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.738267 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:01.738481 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:01.738557 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:01.738716 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:01.738760 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:01.738816 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:01.738880 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:20:01.738906 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:20:01.739869 27759 raft_consensus.cc:1261] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Refusing update from remote peer 83b32205b2804570af172f289ea2276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.740103 27314 raft_consensus.cc:1261] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Refusing update from remote peer 83b32205b2804570af172f289ea2276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:01.740216 27991 consensus_queue.cc:1035] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.740430 27898 consensus_queue.cc:1035] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:01.745887 27071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:01.746016 27071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:01.746927 27071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Sending DeleteTablet for 3 replicas of tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68
I20260810 07:20:01.747274 27285 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:33430
I20260810 07:20:01.747298 27502 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:43484
I20260810 07:20:01.747350 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:43484
I20260810 07:20:01.747386 27923 tablet_replica.cc:330] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: stopping tablet replica
I20260810 07:20:01.747409 27920 tablet_replica.cc:330] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: stopping tablet replica
I20260810 07:20:01.747438 27923 raft_consensus.cc:2229] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:01.747426 27728 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:01 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57974
I20260810 07:20:01.747462 27920 raft_consensus.cc:2229] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:01.747488 27923 raft_consensus.cc:2258] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.747514 27922 tablet_replica.cc:330] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: stopping tablet replica
I20260810 07:20:01.747535 27920 raft_consensus.cc:2258] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.747565 27922 raft_consensus.cc:2229] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:01.747623 27922 raft_consensus.cc:2258] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:01.747972 27920 ts_tablet_manager.cc:1905] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.748000 27923 ts_tablet_manager.cc:1905] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.748162 27071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:01.748332 27071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:01.748832 27922 ts_tablet_manager.cc:1905] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:01.749687 27923 ts_tablet_manager.cc:1918] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:01.749742 27923 log.cc:1196] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/83a2e5ef6f414c04a0c7ef36ae8d1b68
I20260810 07:20:01.749753 27920 ts_tablet_manager.cc:1918] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:01.749825 27920 log.cc:1196] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/83a2e5ef6f414c04a0c7ef36ae8d1b68
I20260810 07:20:01.749955 27923 ts_tablet_manager.cc:1939] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting consensus metadata
I20260810 07:20:01.750077 27920 ts_tablet_manager.cc:1939] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P 83b32205b2804570af172f289ea2276b: Deleting consensus metadata
I20260810 07:20:01.750530 27922 ts_tablet_manager.cc:1918] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:01.750599 27922 log.cc:1196] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/83a2e5ef6f414c04a0c7ef36ae8d1b68
I20260810 07:20:01.750658 27045 catalog_manager.cc:4841] TS a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285): tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 (table test-table [id=9399002f7fd74cbeabc40d42c62a9d07]) successfully deleted
I20260810 07:20:01.750840 27922 ts_tablet_manager.cc:1939] T 83a2e5ef6f414c04a0c7ef36ae8d1b68 P c101f1ee7a6342289017fefa67104457: Deleting consensus metadata
I20260810 07:20:01.751183 27049 catalog_manager.cc:4841] TS c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759): tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 (table test-table [id=9399002f7fd74cbeabc40d42c62a9d07]) successfully deleted
I20260810 07:20:01.751362 27049 catalog_manager.cc:4841] TS 83b32205b2804570af172f289ea2276b (127.14.164.2:43609): tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 (table test-table [id=9399002f7fd74cbeabc40d42c62a9d07]) successfully deleted
W20260810 07:20:01.751446 27049 catalog_manager.cc:4808] TS 83b32205b2804570af172f289ea2276b (127.14.164.2:43609): delete failed for tablet 83a2e5ef6f414c04a0c7ef36ae8d1b68 because the tablet was not found. No further retry: Not found: Tablet not found: 83a2e5ef6f414c04a0c7ef36ae8d1b68
I20260810 07:20:01.751814 27728 tablet_service.cc:1467] Processing CreateTablet for tablet 444a04f13ba24882a60044752d415900 (DEFAULT_TABLE table=test-table [id=c733dfac1b0b47cba950663079cb585a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.751816 27504 tablet_service.cc:1467] Processing CreateTablet for tablet 444a04f13ba24882a60044752d415900 (DEFAULT_TABLE table=test-table [id=c733dfac1b0b47cba950663079cb585a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.751937 27728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444a04f13ba24882a60044752d415900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.751937 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444a04f13ba24882a60044752d415900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.751917 27285 tablet_service.cc:1467] Processing CreateTablet for tablet 444a04f13ba24882a60044752d415900 (DEFAULT_TABLE table=test-table [id=c733dfac1b0b47cba950663079cb585a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:01.752028 27285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444a04f13ba24882a60044752d415900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:01.753309 27888 tablet_bootstrap.cc:492] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: Bootstrap starting.
I20260810 07:20:01.753338 27887 tablet_bootstrap.cc:492] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Bootstrap starting.
I20260810 07:20:01.753338 27891 tablet_bootstrap.cc:492] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Bootstrap starting.
I20260810 07:20:01.753690 27891 tablet_bootstrap.cc:654] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.753669 27887 tablet_bootstrap.cc:654] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.753954 27888 tablet_bootstrap.cc:654] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:01.754388 27887 tablet_bootstrap.cc:492] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: No bootstrap required, opened a new log
I20260810 07:20:01.754441 27887 ts_tablet_manager.cc:1397] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:01.754565 27887 raft_consensus.cc:348] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.754663 27887 raft_consensus.cc:374] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.754693 27887 raft_consensus.cc:729] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b32205b2804570af172f289ea2276b, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.754753 27887 consensus_queue.cc:260] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.754881 27887 ts_tablet_manager.cc:1428] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.755226 27891 tablet_bootstrap.cc:492] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: No bootstrap required, opened a new log
I20260810 07:20:01.755285 27891 ts_tablet_manager.cc:1397] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:20:01.755390 27888 tablet_bootstrap.cc:492] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: No bootstrap required, opened a new log
I20260810 07:20:01.755436 27888 ts_tablet_manager.cc:1397] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:01.755416 27891 raft_consensus.cc:348] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.755503 27891 raft_consensus.cc:374] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.755522 27891 raft_consensus.cc:729] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c101f1ee7a6342289017fefa67104457, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.755584 27891 consensus_queue.cc:260] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.755597 27888 raft_consensus.cc:348] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.755668 27888 raft_consensus.cc:374] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:01.755688 27888 raft_consensus.cc:729] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7fd8b48ad4ccc9d34232d0a66838f, State: Initialized, Role: FOLLOWER
I20260810 07:20:01.755681 27891 ts_tablet_manager.cc:1428] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.755733 27888 consensus_queue.cc:260] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.755818 27888 ts_tablet_manager.cc:1428] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:01.987080 27991 raft_consensus.cc:482] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:01.987198 27991 raft_consensus.cc:504] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.987432 27991 leader_election.cc:288] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:01.987731 27759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444a04f13ba24882a60044752d415900" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c101f1ee7a6342289017fefa67104457" is_pre_election: true
I20260810 07:20:01.987862 27759 raft_consensus.cc:2454] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 0.
I20260810 07:20:01.987787 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444a04f13ba24882a60044752d415900" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" is_pre_election: true
I20260810 07:20:01.987926 27314 raft_consensus.cc:2454] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 0.
I20260810 07:20:01.988150 27431 leader_election.cc:302] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, a5e7fd8b48ad4ccc9d34232d0a66838f; no voters:
I20260810 07:20:01.988325 27991 raft_consensus.cc:2781] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:01.988375 27991 raft_consensus.cc:482] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:01.988394 27991 raft_consensus.cc:3037] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.989444 27991 raft_consensus.cc:504] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.989624 27991 leader_election.cc:288] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 election: Requested vote from peers a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285), c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759)
I20260810 07:20:01.989799 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444a04f13ba24882a60044752d415900" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f"
I20260810 07:20:01.989845 27759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444a04f13ba24882a60044752d415900" candidate_uuid: "83b32205b2804570af172f289ea2276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c101f1ee7a6342289017fefa67104457"
I20260810 07:20:01.989900 27314 raft_consensus.cc:3037] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.989925 27759 raft_consensus.cc:3037] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:01.990724 27759 raft_consensus.cc:2454] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 1.
I20260810 07:20:01.990725 27314 raft_consensus.cc:2454] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b32205b2804570af172f289ea2276b in term 1.
I20260810 07:20:01.990984 27438 leader_election.cc:302] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b32205b2804570af172f289ea2276b, c101f1ee7a6342289017fefa67104457; no voters:
I20260810 07:20:01.991099 27991 raft_consensus.cc:2781] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:01.991199 27991 raft_consensus.cc:686] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 1 LEADER]: Becoming Leader. State: Replica: 83b32205b2804570af172f289ea2276b, State: Running, Role: LEADER
I20260810 07:20:01.991263 27991 consensus_queue.cc:237] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } }
I20260810 07:20:01.992010 27070 catalog_manager.cc:5495] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b reported cstate change: term changed from 0 to 1, leader changed from <none> to 83b32205b2804570af172f289ea2276b (127.14.164.2). New cstate: current_term: 1 leader_uuid: "83b32205b2804570af172f289ea2276b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83b32205b2804570af172f289ea2276b" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 43609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:02.048238 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:02.045621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:33667 (local address 127.14.164.3:37759)
0810 07:20:02.045730 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:02.045732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:02.045858 (+ 126us) server_negotiation.cc:407] Connection header received
0810 07:20:02.045887 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:02.045887 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:02.045898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:02.046015 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:02.046142 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.046625 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.047432 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.047627 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.047684 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:02.047945 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:02.047978 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:02.048036 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:02.048153 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 07:20:02.048172 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 07:20:02.048267 28017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:02.045561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:37759 (local address 127.14.164.2:33667)
0810 07:20:02.045760 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:02.045773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:02.045844 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:02.046014 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:02.046015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:02.046017 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:02.046072 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:02.046073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.046746 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.046747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:02.047348 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:02.047350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.047722 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.047723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:02.047778 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:02.047795 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:02.048076 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:02.048079 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:02.048140 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:20:02.048194 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":129,"threads_started":1}
I20260810 07:20:02.050592 27759 raft_consensus.cc:1261] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Refusing update from remote peer 83b32205b2804570af172f289ea2276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:02.050593 27314 raft_consensus.cc:1261] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Refusing update from remote peer 83b32205b2804570af172f289ea2276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:02.050895 27991 consensus_queue.cc:1035] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7fd8b48ad4ccc9d34232d0a66838f" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:02.050992 27999 consensus_queue.cc:1035] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c101f1ee7a6342289017fefa67104457" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:02.058661 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:02.056324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:34145 (local address 127.14.164.2:43609)
0810 07:20:02.056365 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:02.056367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:02.056396 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:20:02.056474 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:02.056475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:02.056494 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:02.056548 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:02.056743 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.057310 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.058018 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.058197 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.058248 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:02.058415 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:02.058442 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:02.058475 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:02.058556 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 07:20:02.058581 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:20:02.058701 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:02.056229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:43609 (local address 127.14.164.3:34145)
0810 07:20:02.056345 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:02.056359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:02.056414 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:02.056600 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:02.056600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:02.056602 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:02.056648 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:02.056650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.057376 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.057377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:02.057946 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:02.057948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:02.058282 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:02.058284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:02.058337 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:02.058344 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:02.058492 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:02.058494 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:20:02.058551 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 07:20:02.058613 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 07:20:02.065559 27070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:02.065675 27070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:02.066357 27070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0648550a65b45c3aa9626b9c15e25c9: Sending DeleteTablet for 3 replicas of tablet 444a04f13ba24882a60044752d415900
I20260810 07:20:02.066638 27285 tablet_service.cc:1514] Processing DeleteTablet for tablet 444a04f13ba24882a60044752d415900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:02 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:33430
I20260810 07:20:02.066692 27728 tablet_service.cc:1514] Processing DeleteTablet for tablet 444a04f13ba24882a60044752d415900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:02 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57974
I20260810 07:20:02.066726 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet 444a04f13ba24882a60044752d415900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:02 UTC) from {username='oryx', principal='oryx/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:43484
I20260810 07:20:02.066751 27923 tablet_replica.cc:330] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: stopping tablet replica
I20260810 07:20:02.066818 27923 raft_consensus.cc:2229] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:02.066826 27920 tablet_replica.cc:330] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: stopping tablet replica
I20260810 07:20:02.066864 27923 raft_consensus.cc:2258] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:02.066900 27920 raft_consensus.cc:2229] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:02.066943 27922 tablet_replica.cc:330] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: stopping tablet replica
I20260810 07:20:02.066973 27920 raft_consensus.cc:2258] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:02.067018 27922 raft_consensus.cc:2229] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:02.067052 27922 raft_consensus.cc:2258] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:02.067305 27923 ts_tablet_manager.cc:1905] T 444a04f13ba24882a60044752d415900 P a5e7fd8b48ad4ccc9d34232d0a66838f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:02.067396 27920 ts_tablet_manager.cc:1905] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:02.067395 27922 ts_tablet_manager.cc:1905] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:02.068882 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 27186
I20260810 07:20:02.068960 27922 ts_tablet_manager.cc:1918] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:02.068960 27920 ts_tablet_manager.cc:1918] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:02.069025 27922 log.cc:1196] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/444a04f13ba24882a60044752d415900
I20260810 07:20:02.069025 27920 log.cc:1196] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/444a04f13ba24882a60044752d415900
I20260810 07:20:02.069255 27922 ts_tablet_manager.cc:1939] T 444a04f13ba24882a60044752d415900 P c101f1ee7a6342289017fefa67104457: Deleting consensus metadata
I20260810 07:20:02.069255 27920 ts_tablet_manager.cc:1939] T 444a04f13ba24882a60044752d415900 P 83b32205b2804570af172f289ea2276b: Deleting consensus metadata
I20260810 07:20:02.073673 27049 catalog_manager.cc:4841] TS 83b32205b2804570af172f289ea2276b (127.14.164.2:43609): tablet 444a04f13ba24882a60044752d415900 (table test-table [id=c733dfac1b0b47cba950663079cb585a]) successfully deleted
I20260810 07:20:02.077404 27049 catalog_manager.cc:4841] TS c101f1ee7a6342289017fefa67104457 (127.14.164.3:37759): tablet 444a04f13ba24882a60044752d415900 (table test-table [id=c733dfac1b0b47cba950663079cb585a]) successfully deleted
W20260810 07:20:02.083591 27045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:44285 (error 108)
I20260810 07:20:02.083811 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 27408
I20260810 07:20:02.092999 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:02.084151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44285 (local address 127.0.0.1:33446)
0810 07:20:02.092824 (+ 8673us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:02.092925 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:44285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8642,"thread_start_us":42,"threads_started":1}
W20260810 07:20:02.103675 27045 catalog_manager.cc:4584] TS a5e7fd8b48ad4ccc9d34232d0a66838f (127.14.164.1:44285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 444a04f13ba24882a60044752d415900: Network error: Client connection negotiation failed: client connection to 127.14.164.1:44285: connect: Connection refused (error 111)
I20260810 07:20:02.117166 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 27632
I20260810 07:20:02.133960 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 27023
2026-08-10T07:20:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3416 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:02 dist-test-slave-x5fp krb5kdc[28029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:02 dist-test-slave-x5fp krb5kdc[28029](info): set up 2 sockets
Aug 10 07:20:02 dist-test-slave-x5fp krb5kdc[28029](info): commencing operation
krb5kdc: starting...
W20260810 07:20:03.407706 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:20:03 dist-test-slave-x5fp krb5kdc[28029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:20:03.549585 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35305
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:03.639950 28050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:03.640232 28050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:03.640275 28050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:03.641571 28050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:03.641642 28050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:03.641667 28050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:03.641697 28050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:03.641731 28050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:03.643275 28050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:03.643568 28050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:03.643821 28050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:03.649083 28063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:03.649075 28050 server_base.cc:1034] running on GCE node
W20260810 07:20:03.649078 28059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:03.649101 28060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:03.649561 28050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:03.649868 28050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:03.651049 28050 hybrid_clock.cc:648] HybridClock initialized: now 1786346403651030 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T07:20:03Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1511 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:03 dist-test-slave-x5fp krb5kdc[28071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:03 dist-test-slave-x5fp krb5kdc[28071](info): set up 2 sockets
Aug 10 07:20:03 dist-test-slave-x5fp krb5kdc[28071](info): commencing operation
krb5kdc: starting...
W20260810 07:20:04.950120 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:20:05.085083 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36167
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:36167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:05.173753 28101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:05.173995 28101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:05.174021 28101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:05.175266 28101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:05.175318 28101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:05.175333 28101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:05.175346 28101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:05.175359 28101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:05.177038 28101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:36167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:05.177309 28101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:05.177589 28101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:05.183255 28116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.183256 28110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.183255 28111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:05.183295 28101 server_base.cc:1034] running on GCE node
I20260810 07:20:05.183959 28101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:05.184289 28101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:05.185473 28101 hybrid_clock.cc:648] HybridClock initialized: now 1786346405185453 us; error 30 us; skew 500 ppm
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:05.193913 28101 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:20:05.194461 28101 webserver.cc:460] Webserver started at http://127.14.164.62:40249/ using document root <none> and password file <none>
I20260810 07:20:05.194729 28101 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:05.194809 28101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:05.194962 28101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:05.196112 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "225409626c1a400e8a4033f2dd3484d8"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "e8056c7d8fe245b9ae257bea6fa5f9df"
server_key_iv: "203cd1182aa0f895718c7b03ed4aad14"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.196511 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "225409626c1a400e8a4033f2dd3484d8"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "e8056c7d8fe245b9ae257bea6fa5f9df"
server_key_iv: "203cd1182aa0f895718c7b03ed4aad14"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.198060 28101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:20:05.199004 28127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.199227 28101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:05.199340 28101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "225409626c1a400e8a4033f2dd3484d8"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "e8056c7d8fe245b9ae257bea6fa5f9df"
server_key_iv: "203cd1182aa0f895718c7b03ed4aad14"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.199427 28101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:05.224575 28101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:05.227176 28101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:05.227391 28101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:05.231967 28101 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:36167
I20260810 07:20:05.232035 28205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:36167 every 8 connection(s)
I20260810 07:20:05.232470 28101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:05.233455 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:05.235881 28206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Bootstrap starting.
I20260810 07:20:05.236481 28206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:05.236770 28206 log.cc:824] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:05.238206 28206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: No bootstrap required, opened a new log
I20260810 07:20:05.240060 28206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:05.240240 28206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:05.240265 28206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225409626c1a400e8a4033f2dd3484d8, State: Initialized, Role: FOLLOWER
I20260810 07:20:05.240391 28206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:05.240477 28206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:05.240520 28206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:05.240573 28206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:05.241295 28206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:05.241427 28206 leader_election.cc:302] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 225409626c1a400e8a4033f2dd3484d8; no voters:
I20260810 07:20:05.241647 28206 leader_election.cc:288] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:05.241802 28210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:05.242022 28210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 LEADER]: Becoming Leader. State: Replica: 225409626c1a400e8a4033f2dd3484d8, State: Running, Role: LEADER
I20260810 07:20:05.242034 28206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:05.242146 28210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:05.242537 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 28101
I20260810 07:20:05.242689 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:20:05.242918 14992 external_mini_cluster.cc:1428] Setting key c22f4657a5c86f93840f51c0458fd3f5
I20260810 07:20:05.243610 28214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 225409626c1a400e8a4033f2dd3484d8. Latest consensus state: current_term: 1 leader_uuid: "225409626c1a400e8a4033f2dd3484d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } } }
I20260810 07:20:05.243944 28214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:05.243570 28211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "225409626c1a400e8a4033f2dd3484d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } } }
I20260810 07:20:05.244146 28211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:05.244288 28231 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:05.244364 28231 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:05.244451 28235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:05.244699 28235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:05.246820 28235 catalog_manager.cc:1347] Generated new cluster ID: ee5256e2fc6b49a2b2354a7088de0211
I20260810 07:20:05.246881 28235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:05.256562 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.243659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.14.164.62:36167)
0810 07:20:05.243808 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:05.243819 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:05.243973 (+ 154us) server_negotiation.cc:407] Connection header received
0810 07:20:05.244184 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:05.244207 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:05.244283 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:05.244438 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:05.245161 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.246403 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.247413 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.248288 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.248445 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.252487 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:05.252492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:05.252492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:05.252507 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:05.254652 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.255146 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.255150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.255151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.255291 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.255533 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.255533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.255534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.255878 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:05.256035 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:05.256177 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 07:20:05.256328 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260810 07:20:05.267084 28235 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:05.267813 28235 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:05.271108 28235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Generated new TSK 0
I20260810 07:20:05.271350 28235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:20:05.308913 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:36167
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:05.402361 28241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:05.402642 28241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:05.402673 28241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:05.404063 28241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:05.404115 28241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:05.404145 28241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:20:05.405833 28241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:05.406076 28241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:05.406292 28241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:05.411738 28252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.411849 28258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.411994 28253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:05.412051 28241 server_base.cc:1034] running on GCE node
I20260810 07:20:05.412293 28241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:05.412600 28241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:05.413775 28241 hybrid_clock.cc:648] HybridClock initialized: now 1786346405413750 us; error 43 us; skew 500 ppm
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:05.417053 28241 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:20:05.417523 28241 webserver.cc:460] Webserver started at http://127.14.164.1:46637/ using document root <none> and password file <none>
I20260810 07:20:05.417754 28241 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:05.417797 28241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:05.417906 28241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:05.419004 28241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "05d22b5236da4245a7603c22f72c477c"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "71f157ea826f771a5be5a083e6c71f8e"
server_key_iv: "bfb8db61b1c42b9feb3722e72433063b"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.419443 28241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "05d22b5236da4245a7603c22f72c477c"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "71f157ea826f771a5be5a083e6c71f8e"
server_key_iv: "bfb8db61b1c42b9feb3722e72433063b"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.421082 28241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:05.421878 28271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.422049 28241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:20:05.422155 28241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "05d22b5236da4245a7603c22f72c477c"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "71f157ea826f771a5be5a083e6c71f8e"
server_key_iv: "bfb8db61b1c42b9feb3722e72433063b"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.422237 28241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:05.436515 28241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:05.439076 28241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:05.439257 28241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:05.439530 28241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:05.439911 28241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:05.439967 28241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.440004 28241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:05.440048 28241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.446473 28241 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:40585
I20260810 07:20:05.446523 28452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:40585 every 8 connection(s)
I20260810 07:20:05.446872 28241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:05.455881 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 28241
I20260810 07:20:05.456017 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:05.456229 14992 external_mini_cluster.cc:1428] Setting key 5bdb7dc0a8455d3071cf8aa9cced35a4
I20260810 07:20:05.456319 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.447749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:49019 (local address 127.14.164.62:36167)
0810 07:20:05.447804 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:05.447806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:05.447874 (+ 68us) server_negotiation.cc:407] Connection header received
0810 07:20:05.448395 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:05.448396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:05.448409 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:05.448479 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:05.449473 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.449979 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.450975 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.451189 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.451250 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.453558 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:05.453559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:05.453560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:05.453567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:05.454823 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.455208 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.455209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.455209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.455274 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.455486 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.455487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.455487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.455601 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:05.455653 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:05.456092 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 07:20:05.456211 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:20:05.456454 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.447584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:49019)
0810 07:20:05.447750 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:05.447796 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:05.448296 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:05.448537 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:05.448541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:05.449176 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:05.449388 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:05.449394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.450064 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.450065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:05.450884 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:05.450886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.451268 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.451269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:05.451373 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.451956 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:05.451966 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:05.453466 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:05.454907 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:05.454914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:05.454916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:05.455142 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:05.455333 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:05.455334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:05.455334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:05.455434 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:05.455678 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:05.455684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:05.455883 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 07:20:05.456015 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 07:20:05.456199 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260810 07:20:05.457136 28453 heartbeater.cc:347] Connected to a master server at 127.14.164.62:36167
I20260810 07:20:05.457231 28453 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:05.457430 28453 heartbeater.cc:511] Master 127.14.164.62:36167 requested a full tablet report, sending...
I20260810 07:20:05.457993 28154 ts_manager.cc:194] Registered new tserver with Master: 05d22b5236da4245a7603c22f72c477c (127.14.164.1:40585)
I20260810 07:20:05.458657 28154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:49019
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:20:05.488543 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:36167
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:05.578754 28462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:05.579003 28462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:05.579031 28462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:05.580272 28462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:05.580322 28462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:05.580349 28462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:20:05.581965 28462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:05.582250 28462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:05.582496 28462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:05.588068 28468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.588068 28474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.588091 28469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:05.588459 28462 server_base.cc:1034] running on GCE node
I20260810 07:20:05.588702 28462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:05.589008 28462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:05.590217 28462 hybrid_clock.cc:648] HybridClock initialized: now 1786346405590208 us; error 42 us; skew 500 ppm
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:05.593165 28462 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:20:05.593672 28462 webserver.cc:460] Webserver started at http://127.14.164.2:35935/ using document root <none> and password file <none>
I20260810 07:20:05.593931 28462 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:05.593999 28462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:05.594151 28462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:05.595273 28462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "abc6ef68f3a54e458ec61bfb0bba912d"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "9588fa4fc0a8e74c7494e1a45f0941a8"
server_key_iv: "25a457c1deae9535f1ffeefb22b6fde8"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.595664 28462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "abc6ef68f3a54e458ec61bfb0bba912d"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "9588fa4fc0a8e74c7494e1a45f0941a8"
server_key_iv: "25a457c1deae9535f1ffeefb22b6fde8"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.597303 28462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:20:05.598201 28483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.598410 28462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:05.598532 28462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "abc6ef68f3a54e458ec61bfb0bba912d"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "9588fa4fc0a8e74c7494e1a45f0941a8"
server_key_iv: "25a457c1deae9535f1ffeefb22b6fde8"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.598611 28462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:05.611402 28462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:05.614178 28462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:05.614398 28462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:05.614668 28462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:05.615016 28462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:05.615070 28462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.615108 28462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:05.615149 28462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.621423 28462 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:34303
I20260810 07:20:05.621472 28675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:34303 every 8 connection(s)
I20260810 07:20:05.621827 28462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:05.626173 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 28462
I20260810 07:20:05.626281 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:05.626538 14992 external_mini_cluster.cc:1428] Setting key bfa2d065ea82cd665ebecb8e75236b82
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:05.633334 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.622664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:37239 (local address 127.14.164.62:36167)
0810 07:20:05.622759 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:05.622761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:05.622980 (+ 219us) server_negotiation.cc:407] Connection header received
0810 07:20:05.623612 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:05.623614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:05.623626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:05.623707 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:05.624880 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.625484 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.626712 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.626973 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.627065 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.630180 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:05.630182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:05.630182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:05.630192 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:05.631697 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.632187 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.632189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.632190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.632248 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.632550 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.632551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.632551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.632689 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:05.632794 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:05.633132 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 07:20:05.633231 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:20:05.633555 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.622641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:37239)
0810 07:20:05.622836 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:05.622885 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:05.623443 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:05.623825 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:05.623830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:05.624490 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:05.624773 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:05.624782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.625586 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.625587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:05.626628 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:05.626632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.627130 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.627132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:05.627269 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.627939 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:05.627950 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:05.630063 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:05.631809 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:05.631819 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:05.631822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:05.632090 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:05.632322 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:05.632323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:05.632323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:05.632474 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:05.632796 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:05.632805 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:05.632975 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 07:20:05.633097 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:20:05.633286 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260810 07:20:05.634505 28679 heartbeater.cc:347] Connected to a master server at 127.14.164.62:36167
I20260810 07:20:05.634618 28679 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:05.634872 28679 heartbeater.cc:511] Master 127.14.164.62:36167 requested a full tablet report, sending...
I20260810 07:20:05.635385 28154 ts_manager.cc:194] Registered new tserver with Master: abc6ef68f3a54e458ec61bfb0bba912d (127.14.164.2:34303)
I20260810 07:20:05.635789 28154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:37239
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:20:05.660230 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:36167
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:05.756541 28689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:05.756843 28689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:05.756875 28689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:05.758106 28689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:05.758152 28689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:05.758178 28689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:20:05.759680 28689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:05.759961 28689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:05.760254 28689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:05.765698 28697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.765700 28703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:05.765935 28701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:05.765955 28689 server_base.cc:1034] running on GCE node
I20260810 07:20:05.766263 28689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:05.766561 28689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:05.767735 28689 hybrid_clock.cc:648] HybridClock initialized: now 1786346405767733 us; error 43 us; skew 500 ppm
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:05.770514 28689 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:20:05.771093 28689 webserver.cc:460] Webserver started at http://127.14.164.3:43437/ using document root <none> and password file <none>
I20260810 07:20:05.771347 28689 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:05.771425 28689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:05.771577 28689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:05.772707 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "54f7fd2a1f04471d88a4b63126ebde60"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "139df5f0a78c049a574160c1ff7f00a5"
server_key_iv: "f8fdea0a1c352864663350cc9fc824a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.773161 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "54f7fd2a1f04471d88a4b63126ebde60"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "139df5f0a78c049a574160c1ff7f00a5"
server_key_iv: "f8fdea0a1c352864663350cc9fc824a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.774634 28689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:05.775540 28717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.775743 28689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:05.775887 28689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "54f7fd2a1f04471d88a4b63126ebde60"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "139df5f0a78c049a574160c1ff7f00a5"
server_key_iv: "f8fdea0a1c352864663350cc9fc824a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:05.775970 28689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:05.786608 28689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:05.788997 28689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:05.789170 28689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:05.789441 28689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:05.789793 28689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:05.789846 28689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.789891 28689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:05.789921 28689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:05.796200 28689 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:41795
I20260810 07:20:05.796254 28904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:41795 every 8 connection(s)
I20260810 07:20:05.796627 28689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:05.797063 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 28689
I20260810 07:20:05.797165 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:05.797343 14992 external_mini_cluster.cc:1428] Setting key 39b7dfda8da62eb07d6b4aebd5552a8f
Aug 10 07:20:05 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346405, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:05.806627 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.797445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:46825 (local address 127.14.164.62:36167)
0810 07:20:05.797518 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:05.797520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:05.797713 (+ 193us) server_negotiation.cc:407] Connection header received
0810 07:20:05.798335 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:05.798336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:05.798348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:05.798410 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:05.799375 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.799870 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.800950 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.801155 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.801220 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.803669 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:05.803671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:05.803671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:05.803678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:05.805214 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.805632 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.805633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.805633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.805688 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:05.806001 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:05.806001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:05.806002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:05.806125 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:05.806179 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:05.806488 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 07:20:05.806556 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:20:05.806830 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:05.797399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:46825)
0810 07:20:05.797590 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:05.797632 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:05.798251 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:05.798468 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:05.798472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:05.799075 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:05.799276 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:05.799281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.799956 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.799957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:05.800864 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:05.800867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:05.801240 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:05.801241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:05.801360 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:05.801921 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:05.801930 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:05.803576 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:05.805314 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:05.805322 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:05.805324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:05.805561 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:05.805766 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:05.805768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:05.805768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:05.805937 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:05.806207 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:05.806214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:05.806354 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 07:20:05.806460 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 07:20:05.806599 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260810 07:20:05.807459 28905 heartbeater.cc:347] Connected to a master server at 127.14.164.62:36167
I20260810 07:20:05.807580 28905 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:05.807793 28905 heartbeater.cc:511] Master 127.14.164.62:36167 requested a full tablet report, sending...
I20260810 07:20:05.808247 28154 ts_manager.cc:194] Registered new tserver with Master: 54f7fd2a1f04471d88a4b63126ebde60 (127.14.164.3:41795)
I20260810 07:20:05.808633 28154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:46825
I20260810 07:20:05.809155 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab.
Entry for principal oryx/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab.
WARNING: no policy specified for oryx/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab.
Entry for principal oryx/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab.
WARNING: no policy specified for oryx/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab.
Entry for principal oryx/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab.
I20260810 07:20:05.891222 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 28241
I20260810 07:20:05.910331 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 28462
I20260810 07:20:05.922148 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 28689
I20260810 07:20:05.940595 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 28101
I20260810 07:20:05.951170 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36167
--webserver_interface=127.14.164.62
--webserver_port=40249
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:36167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:06.040091 28920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:06.040303 28920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:06.040328 28920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:06.041589 28920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:06.041636 28920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:06.041651 28920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:06.041663 28920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:06.041676 28920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:06.043211 28920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:36167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=40249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:06.043480 28920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:06.043717 28920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:06.049335 28939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.049363 28929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.049387 28920 server_base.cc:1034] running on GCE node
W20260810 07:20:06.049335 28927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.050035 28920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:06.050436 28920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:06.051630 28920 hybrid_clock.cc:648] HybridClock initialized: now 1786346406051592 us; error 50 us; skew 500 ppm
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:06.054678 28920 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:20:06.055178 28920 webserver.cc:460] Webserver started at http://127.14.164.62:40249/ using document root <none> and password file <none>
I20260810 07:20:06.055434 28920 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:06.055528 28920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:06.057065 28920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:20:06.057809 28949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.057986 28920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:06.058094 28920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "225409626c1a400e8a4033f2dd3484d8"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "e8056c7d8fe245b9ae257bea6fa5f9df"
server_key_iv: "203cd1182aa0f895718c7b03ed4aad14"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.058501 28920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:06.079095 28920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:06.081699 28920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:06.081875 28920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:06.086133 28920 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:36167
I20260810 07:20:06.086190 29032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:36167 every 8 connection(s)
I20260810 07:20:06.086613 28920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:06.087599 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 28920
I20260810 07:20:06.087972 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:40585
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:36167
--webserver_port=46637
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:20:06.088554 29033 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:20:06.089375 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Bootstrap starting.
I20260810 07:20:06.101186 29033 log.cc:824] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:06.103579 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:20:06.103937 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Bootstrap complete.
I20260810 07:20:06.107403 29033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:06.107762 29033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225409626c1a400e8a4033f2dd3484d8, State: Initialized, Role: FOLLOWER
I20260810 07:20:06.107942 29033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:06.108011 29033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:06.108059 29033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:06.108107 29033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:20:06.109593 29033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:06.109732 29033 leader_election.cc:302] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 225409626c1a400e8a4033f2dd3484d8; no voters:
I20260810 07:20:06.109913 29033 leader_election.cc:288] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:20:06.110020 29037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:20:06.110195 29033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:06.110214 29037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [term 2 LEADER]: Becoming Leader. State: Replica: 225409626c1a400e8a4033f2dd3484d8, State: Running, Role: LEADER
I20260810 07:20:06.110348 29037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } }
I20260810 07:20:06.110642 29039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "225409626c1a400e8a4033f2dd3484d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } } }
I20260810 07:20:06.110738 29039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:06.111029 29040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 225409626c1a400e8a4033f2dd3484d8. Latest consensus state: current_term: 2 leader_uuid: "225409626c1a400e8a4033f2dd3484d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "225409626c1a400e8a4033f2dd3484d8" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36167 } } }
I20260810 07:20:06.111182 29040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:06.111194 29046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:06.111887 29046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:06.112044 29046 catalog_manager.cc:1259] Loaded cluster ID: ee5256e2fc6b49a2b2354a7088de0211
I20260810 07:20:06.112267 29046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:06.113606 29046 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:06.113797 29046 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 225409626c1a400e8a4033f2dd3484d8: Loaded TSK: 0
I20260810 07:20:06.114013 29046 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:20:06.184661 29035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:06.184978 29035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:06.185029 29035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:06.186249 29035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:06.186316 29035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:06.186362 29035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:20:06.187842 29035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.1.keytab
--principal=oryx/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:40585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=46637
--tserver_master_addrs=127.14.164.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:06.188127 29035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:06.188367 29035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:06.193696 29077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.193638 29073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.193629 29074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.194651 29035 server_base.cc:1034] running on GCE node
I20260810 07:20:06.194905 29035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:06.195245 29035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:06.196442 29035 hybrid_clock.cc:648] HybridClock initialized: now 1786346406196417 us; error 46 us; skew 500 ppm
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:06.199695 29035 init.cc:377] Logged in from keytab as oryx/127.14.164.1@KRBTEST.COM (short username oryx)
I20260810 07:20:06.200224 29035 webserver.cc:460] Webserver started at http://127.14.164.1:46637/ using document root <none> and password file <none>
I20260810 07:20:06.200477 29035 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:06.200565 29035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:06.202204 29035 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:06.202941 29087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.203095 29035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:06.203223 29035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "05d22b5236da4245a7603c22f72c477c"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "71f157ea826f771a5be5a083e6c71f8e"
server_key_iv: "bfb8db61b1c42b9feb3722e72433063b"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.203604 29035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:06.227308 29035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:06.229837 29035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:06.230012 29035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:06.230300 29035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:06.230655 29035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:06.230692 29035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.230719 29035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:06.230767 29035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.237812 29035 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:40585
I20260810 07:20:06.237872 29272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:40585 every 8 connection(s)
I20260810 07:20:06.238267 29035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:06.245997 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 29035
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.246345 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:34303
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:36167
--webserver_port=35935
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.248281 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.239238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:49577)
0810 07:20:06.239439 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.239481 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.240208 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.240734 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.240739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.241425 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.241634 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.241639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.242853 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.242854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.243762 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.243765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.244427 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.244428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.244571 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.245217 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.245227 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.248032 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
W20260810 07:20:06.248461 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.239526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:49577 (local address 127.14.164.62:36167)
0810 07:20:06.239729 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.239740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.239784 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:20:06.240473 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.240479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.240524 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.240670 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.241764 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.242730 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.243880 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.244310 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.244439 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.248250 (+ 3811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:49577: BlockingRecv error: recv got EOF from 127.14.164.1:49577 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
W20260810 07:20:06.248584 29273 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:36167 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.164.62:36167: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.254144 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.248915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:56509)
0810 07:20:06.249066 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.249080 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.249154 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.249282 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.249283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.249476 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.249555 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.249557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.250189 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.250191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.251110 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.251113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.251486 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.251487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.251563 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.252038 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.252043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.254039 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
W20260810 07:20:06.254192 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.248973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:56509 (local address 127.14.164.62:36167)
0810 07:20:06.249033 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.249035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.249135 (+ 100us) server_negotiation.cc:407] Connection header received
0810 07:20:06.249215 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.249215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.249226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.249277 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.249633 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.250105 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.251197 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.251372 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.251453 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.254123 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:56509: BlockingRecv error: recv got EOF from 127.14.164.1:56509 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.259044 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.254419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:35439)
0810 07:20:06.254463 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.254473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.254505 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.254676 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.254677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.254788 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.254817 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.254818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.255728 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.255729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.256223 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.256224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.256708 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.256709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.256794 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.257237 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.257242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.258946 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 07:20:06.259064 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.254504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:35439 (local address 127.14.164.62:36167)
0810 07:20:06.254550 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.254552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.254561 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:20:06.254585 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.254586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.254599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.254673 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.254888 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.255650 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.256303 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.256632 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.256712 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.259018 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:35439: BlockingRecv error: recv got EOF from 127.14.164.1:35439 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 07:20:06.259197 29273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:20:06.335382 29280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:06.335615 29280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:06.335642 29280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:06.336815 29280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:06.336859 29280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:06.336886 29280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:20:06.338326 29280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.2.keytab
--principal=oryx/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:34303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=35935
--tserver_master_addrs=127.14.164.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:06.338539 29280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:06.338735 29280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:06.344237 29297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.344233 29295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.344254 29293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.344494 29280 server_base.cc:1034] running on GCE node
I20260810 07:20:06.344779 29280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:06.345034 29280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:06.346194 29280 hybrid_clock.cc:648] HybridClock initialized: now 1786346406346163 us; error 45 us; skew 500 ppm
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:06.348907 29280 init.cc:377] Logged in from keytab as oryx/127.14.164.2@KRBTEST.COM (short username oryx)
I20260810 07:20:06.349443 29280 webserver.cc:460] Webserver started at http://127.14.164.2:35935/ using document root <none> and password file <none>
I20260810 07:20:06.349685 29280 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:06.349771 29280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:06.351233 29280 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:06.351949 29304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.352180 29280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:06.352291 29280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "abc6ef68f3a54e458ec61bfb0bba912d"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "9588fa4fc0a8e74c7494e1a45f0941a8"
server_key_iv: "25a457c1deae9535f1ffeefb22b6fde8"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.352692 29280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:06.375257 29280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:06.377810 29280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:06.378039 29280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:06.378324 29280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:06.378679 29280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:06.378732 29280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.378777 29280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:06.378814 29280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.385003 29280 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:34303
I20260810 07:20:06.385064 29488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:34303 every 8 connection(s)
I20260810 07:20:06.385462 29280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:06.393532 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.386305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:48935 (local address 127.14.164.62:36167)
0810 07:20:06.386364 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.386365 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.386592 (+ 227us) server_negotiation.cc:407] Connection header received
0810 07:20:06.387127 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.387129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.387144 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.387210 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.388205 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.388773 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.389798 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.390013 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.390083 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.393470 (+ 3387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:48935: BlockingRecv error: recv got EOF from 127.14.164.2:48935 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:20:06.393565 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.386280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:48935)
0810 07:20:06.386470 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.386508 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.387024 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.387282 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.387286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.387880 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.388105 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.388110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.388868 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.388869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.389695 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.389697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.390106 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.390107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.390240 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.390760 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.390769 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.393283 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
W20260810 07:20:06.393926 29489 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:36167 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.164.62:36167: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
I20260810 07:20:06.394109 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 29280
I20260810 07:20:06.394496 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:41795
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:36167
--webserver_port=43437
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.400017 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.394498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:51391)
0810 07:20:06.394791 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.394834 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.394881 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.395037 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.395038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.395163 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.395251 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.395252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.396165 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.396166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.396717 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.396762 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.397256 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.397257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.397322 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.397818 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.397824 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.399902 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273}
W20260810 07:20:06.400162 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.394554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:51391 (local address 127.14.164.62:36167)
0810 07:20:06.394881 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.394884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.394925 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:20:06.394951 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.394952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.394965 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.395021 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.395331 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.396073 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.396836 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.397144 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.397224 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.400055 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:51391: BlockingRecv error: recv got EOF from 127.14.164.2:51391 (error 108)
Metrics: {"server-negotiator.queue_time_us":289}
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.405723 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.400305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:56173)
0810 07:20:06.400456 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.400466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.400525 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.400677 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.400678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.400862 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.400913 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.400915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.401567 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.401569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.402454 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.402457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.402881 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.402883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.402949 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.403455 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.403460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.405616 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
W20260810 07:20:06.405776 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.400365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:56173 (local address 127.14.164.62:36167)
0810 07:20:06.400428 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.400431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.400527 (+ 96us) server_negotiation.cc:407] Connection header received
0810 07:20:06.400585 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.400586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.400597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.400655 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.401015 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.401487 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.402577 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.402802 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.402864 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.405700 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:56173: BlockingRecv error: recv got EOF from 127.14.164.2:56173 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:20:06.405937 29489 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:20:06.490170 29492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:06.490432 29492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:06.490459 29492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:06.491608 29492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:06.491653 29492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:06.491689 29492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:20:06.493213 29492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/krb5kdc/oryx_127.14.164.3.keytab
--principal=oryx/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:41795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=43437
--tserver_master_addrs=127.14.164.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:06.493439 29492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:06.493641 29492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:06.499439 29502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.499447 29504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.499531 29492 server_base.cc:1034] running on GCE node
W20260810 07:20:06.499447 29506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.499953 29492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:06.500253 29492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:06.501449 29492 hybrid_clock.cc:648] HybridClock initialized: now 1786346406501431 us; error 45 us; skew 500 ppm
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:06.504235 29492 init.cc:377] Logged in from keytab as oryx/127.14.164.3@KRBTEST.COM (short username oryx)
I20260810 07:20:06.504863 29492 webserver.cc:460] Webserver started at http://127.14.164.3:43437/ using document root <none> and password file <none>
I20260810 07:20:06.505115 29492 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:06.505206 29492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:06.506791 29492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:06.507668 29521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.508046 29492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:06.508222 29492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "54f7fd2a1f04471d88a4b63126ebde60"
format_stamp: "Formatted at 2026-08-10 07:20:05 on dist-test-slave-x5fp"
server_key: "139df5f0a78c049a574160c1ff7f00a5"
server_key_iv: "f8fdea0a1c352864663350cc9fc824a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.508692 29492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:06.524389 29492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:06.527179 29492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:06.527370 29492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:06.527710 29492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:06.528082 29492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:06.528143 29492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.528183 29492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:06.528210 29492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.535636 29492 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:41795
I20260810 07:20:06.535701 29706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:41795 every 8 connection(s)
I20260810 07:20:06.536181 29492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:06.542490 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 29492
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:06.545537 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.537233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:44091 (local address 127.14.164.62:36167)
0810 07:20:06.537328 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.537331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.537638 (+ 307us) server_negotiation.cc:407] Connection header received
0810 07:20:06.538144 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.538146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.538157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.538220 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.539336 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.539914 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.541240 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.541462 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.541539 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.545471 (+ 3932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:44091: BlockingRecv error: recv got EOF from 127.14.164.3:44091 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:20:06.545643 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.537223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:44091)
0810 07:20:06.537425 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.537473 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.538014 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.538314 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.538318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.538972 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.539191 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.539196 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.540018 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.540019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.541109 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.541113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.541610 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.541612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.541756 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.542528 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.542539 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.545388 (+ 2849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
W20260810 07:20:06.546156 29707 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:36167 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.164.62:36167: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.552820 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.546456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:34131)
0810 07:20:06.546513 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.546523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.546567 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.546689 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.546690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.546865 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.546936 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.546939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.548166 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.548167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.548904 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.548908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.549574 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.549575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.549683 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.550164 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.550174 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.552672 (+ 2498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:20:06.552891 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.546502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:34131 (local address 127.14.164.62:36167)
0810 07:20:06.546542 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.546544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.546551 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:20:06.546630 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.546630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.546640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.546686 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.547047 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.547836 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.549011 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.549298 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.549579 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.552813 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:34131: BlockingRecv error: recv got EOF from 127.14.164.3:34131 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:20:06.553848 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.543302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.14.164.62:36167)
0810 07:20:06.543464 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.543466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.543474 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:20:06.543503 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.543504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.543514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.543596 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.544506 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.545139 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.546103 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.546390 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.546492 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.548060 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:06.548066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:06.548067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:06.548082 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:06.551164 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:06.551725 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:06.551730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:06.551730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:06.551890 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:06.552527 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:06.552529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:06.552530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:06.553126 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:06.553346 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:06.553598 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 07:20:06.553765 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:06 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:06.559216 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.553303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:52793)
0810 07:20:06.553453 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:06.553465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:06.553509 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:06.553651 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:06.553652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:06.553827 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:06.553871 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.553873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.554813 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.554814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.555651 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:06.555655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.556210 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.556211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:06.556318 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.557038 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:06.557045 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:06.559107 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95}
W20260810 07:20:06.559314 29279 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:06.553414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:52793 (local address 127.14.164.62:36167)
0810 07:20:06.553488 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.553491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.553500 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:20:06.553565 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.553565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.553577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.553647 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.553969 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.554659 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.555767 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.556078 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.556205 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.559241 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:52793: BlockingRecv error: recv got EOF from 127.14.164.3:52793 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 07:20:06.559459 29707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 07:20:07 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:07 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:07.264864 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:07.259663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:36739)
0810 07:20:07.259849 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:07.259861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:07.259925 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:07.260080 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:07.260081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:07.260267 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:07.260322 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:07.260323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.261197 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.261198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:07.261908 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:07.261911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.262305 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.262306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:07.262369 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:07.262889 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:07.262895 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:07.264791 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
W20260810 07:20:07.265017 29720 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:07.259790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:36739 (local address 127.14.164.62:36167)
0810 07:20:07.259925 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:07.259927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:07.259938 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:20:07.259993 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:07.259994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:07.260005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:07.260060 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:07.260414 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.261092 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.261983 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.262197 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.262259 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:07.264912 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:36739: BlockingRecv error: recv got EOF from 127.14.164.1:36739 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Aug 10 07:20:07 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:07 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:07.411543 29721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:07.406379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:56961)
0810 07:20:07.406517 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:07.406536 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:07.406599 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:07.406725 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:07.406727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:07.406909 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:07.406962 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:07.406964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.407668 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.407669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:07.408247 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:07.408249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.408640 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.408642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:07.408695 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:07.409245 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:07.409250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:07.411450 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260810 07:20:07.411736 29720 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:07.406411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:56961 (local address 127.14.164.62:36167)
0810 07:20:07.406463 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:07.406465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:07.406584 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:20:07.406652 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:07.406653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:07.406663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:07.406704 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:07.407031 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.407582 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.408334 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.408548 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.408606 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:07.411631 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:56961: BlockingRecv error: recv got EOF from 127.14.164.2:56961 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 10 07:20:07 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:07 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:07.565771 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:07.560056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:53961)
0810 07:20:07.560318 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:07.560341 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:07.560443 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:07.560657 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:07.560658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:07.560967 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:07.561025 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:07.561027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.562067 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.562069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:07.562751 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:07.562754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.563161 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.563162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:07.563210 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:07.563758 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:07.563763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:07.565697 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
W20260810 07:20:07.565994 29720 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:07.560189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:53961 (local address 127.14.164.62:36167)
0810 07:20:07.560365 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:07.560368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:07.560415 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:20:07.560552 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:07.560554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:07.560575 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:07.560646 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:07.561118 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.561931 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.562870 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:07.563074 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:07.563139 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:07.565909 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:53961: BlockingRecv error: recv got EOF from 127.14.164.3:53961 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 10 07:20:08 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:08 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:08.270769 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.265432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:52573)
0810 07:20:08.265628 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:08.265641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:08.265709 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:08.265923 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:08.265924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:08.266124 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:08.266177 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.266179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.266958 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.266960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.267574 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.267576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.268017 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.268018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.268068 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.268688 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:08.268694 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:08.270700 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
W20260810 07:20:08.271055 29725 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:08.265516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:52573 (local address 127.14.164.62:36167)
0810 07:20:08.265726 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.265729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.265740 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:20:08.265768 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.265770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.265784 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.265880 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.266294 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.266855 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.267668 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.267936 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.268002 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.270952 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:52573: BlockingRecv error: recv got EOF from 127.14.164.1:52573 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
Aug 10 07:20:08 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:08 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:08.418185 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.412184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:34687)
0810 07:20:08.412374 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:08.412386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:08.412469 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:08.412691 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:08.412692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:08.412947 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:08.413044 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.413047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.413865 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.413866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.414464 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.414466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.414944 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.414945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.414997 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.415516 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:08.415522 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:08.418116 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
W20260810 07:20:08.418371 29725 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:08.412236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:34687 (local address 127.14.164.62:36167)
0810 07:20:08.412305 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.412307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.412463 (+ 156us) server_negotiation.cc:407] Connection header received
0810 07:20:08.412583 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.412584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.412605 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.412679 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.413151 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.413784 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.414571 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.414856 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.414928 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.418284 (+ 3356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:34687: BlockingRecv error: recv got EOF from 127.14.164.2:34687 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 07:20:08 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:08 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:08.572305 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.566414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:51359)
0810 07:20:08.566589 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:08.566608 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:08.566689 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:08.566905 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:08.566907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:08.567088 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:08.567143 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.567144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.568347 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.568349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.569118 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.569121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.569560 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.569561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.569637 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.570245 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:08.570252 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:08.572240 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
W20260810 07:20:08.572451 29725 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:08.566641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:51359 (local address 127.14.164.62:36167)
0810 07:20:08.566715 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.566722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.566734 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:08.566767 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.566768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.566782 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.566840 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.567264 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.568190 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.569240 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.569469 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.569558 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.572357 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:51359: BlockingRecv error: recv got EOF from 127.14.164.3:51359 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 07:20:09 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:09 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:09.276420 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:09.271348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:39243)
0810 07:20:09.271492 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:09.271507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:09.271596 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:09.271750 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:09.271751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:09.271942 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:09.271997 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:09.271999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.272694 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.272694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:09.273305 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:09.273306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.273724 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.273725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:09.273768 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:09.274294 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:09.274299 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:09.276342 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
W20260810 07:20:09.276616 29729 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:09.271451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:39243 (local address 127.14.164.62:36167)
0810 07:20:09.271606 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:09.271608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:09.271619 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:20:09.271646 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:09.271647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:09.271658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:09.271721 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:09.272080 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.272612 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.273424 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.273645 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.273715 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:09.276503 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:39243: BlockingRecv error: recv got EOF from 127.14.164.1:39243 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
Aug 10 07:20:09 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:09 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:09.423928 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:09.418726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:59589)
0810 07:20:09.418901 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:09.418915 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:09.418986 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:09.419144 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:09.419145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:09.419381 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:09.419433 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:09.419435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.420169 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.420170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:09.420781 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:09.420783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.421214 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.421215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:09.421266 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:09.421783 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:09.421788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:09.423855 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":147,"threads_started":1}
W20260810 07:20:09.424086 29729 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:09.418882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:59589 (local address 127.14.164.62:36167)
0810 07:20:09.418951 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:09.418953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:09.418976 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:20:09.419043 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:09.419044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:09.419056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:09.419122 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:09.419535 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.420058 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.420928 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.421132 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.421201 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:09.423987 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:59589: BlockingRecv error: recv got EOF from 127.14.164.2:59589 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 07:20:09 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:09 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:09.578284 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:09.573080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:48329)
0810 07:20:09.573272 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:09.573286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:09.573399 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:09.573586 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:09.573588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:09.573783 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:09.573835 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:09.573837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.574560 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.574561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:09.575091 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:09.575092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.575470 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.575471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:09.575535 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:09.576127 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:09.576132 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:09.578170 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
W20260810 07:20:09.578338 29729 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:09.573214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:48329 (local address 127.14.164.62:36167)
0810 07:20:09.573287 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:09.573289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:09.573365 (+ 76us) server_negotiation.cc:407] Connection header received
0810 07:20:09.573470 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:09.573472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:09.573494 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:09.573557 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:09.573932 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.574458 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.575184 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:09.575390 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:09.575471 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:09.578266 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:48329: BlockingRecv error: recv got EOF from 127.14.164.3:48329 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 07:20:10 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:10 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:10.286192 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.277029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:58417)
0810 07:20:10.277183 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:10.277203 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:10.277258 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:10.277405 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:10.277406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:10.277587 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:10.277641 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.277642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.278362 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.278363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.278897 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.278899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.279371 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.279372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.279482 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.284020 (+ 4538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:10.284025 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:10.286090 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260810 07:20:10.286283 29736 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:10.277121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:58417 (local address 127.14.164.62:36167)
0810 07:20:10.277268 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:10.277270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:10.277279 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:20:10.277305 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:10.277306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:10.277317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:10.277391 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:10.277711 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.278254 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.278992 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.279231 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.279302 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.286195 (+ 6893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:58417: BlockingRecv error: recv got EOF from 127.14.164.1:58417 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
Aug 10 07:20:10 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:10 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:10.430295 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.424707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:43277)
0810 07:20:10.424980 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:10.424993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:10.425218 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:10.425425 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:10.425426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:10.425640 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:10.425695 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.425697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.426496 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.426497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.427096 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.427098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.427510 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.427511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.427582 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.428173 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:10.428178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:10.430232 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":77,"threads_started":1}
W20260810 07:20:10.430472 29736 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:10.424841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:43277 (local address 127.14.164.62:36167)
0810 07:20:10.424970 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:10.424979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:10.425187 (+ 208us) server_negotiation.cc:407] Connection header received
0810 07:20:10.425300 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:10.425302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:10.425320 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:10.425406 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:10.425790 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.426399 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.427194 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.427424 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.427511 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.430371 (+ 2860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:43277: BlockingRecv error: recv got EOF from 127.14.164.2:43277 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 07:20:10 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:10 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:10.584918 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.579024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:36033)
0810 07:20:10.579433 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:10.579446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:10.579523 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:10.579749 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:10.579751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:10.580018 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:10.580076 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.580078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.581036 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.581037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.581581 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.581583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.581995 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.581996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.582041 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.582829 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:10.582834 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:10.584842 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":295,"threads_started":1}
W20260810 07:20:10.585060 29736 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:10.579190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:36033 (local address 127.14.164.62:36167)
0810 07:20:10.579320 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:10.579323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:10.579527 (+ 204us) server_negotiation.cc:407] Connection header received
0810 07:20:10.579629 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:10.579630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:10.579653 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:10.579718 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:10.580179 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.580917 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.581685 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.581917 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.581982 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.584972 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:36033: BlockingRecv error: recv got EOF from 127.14.164.3:36033 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 10 07:20:11 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:11 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:11.292217 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.286799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:54543)
0810 07:20:11.286961 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:11.286972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:11.287025 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:11.287212 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:11.287213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:11.287405 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:11.287459 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.287460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.288110 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.288111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.288631 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.288633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.289043 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.289044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.289089 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.289744 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:11.289750 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:11.292095 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
W20260810 07:20:11.292376 29743 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:11.286931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:54543 (local address 127.14.164.62:36167)
0810 07:20:11.287069 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:11.287072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:11.287080 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:20:11.287105 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:11.287105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:11.287116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:11.287186 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:11.287530 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.288034 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.288752 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.288966 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.289030 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.292242 (+ 3212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:54543: BlockingRecv error: recv got EOF from 127.14.164.1:54543 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
Aug 10 07:20:11 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:11 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:11.436096 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.430897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:51367)
0810 07:20:11.431158 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:11.431172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:11.431239 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:11.431488 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:11.431489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:11.431667 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:11.431720 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.431721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.432449 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.432450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.433037 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.433040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.433397 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.433398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.433443 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.433991 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:11.434005 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:11.436032 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":67,"threads_started":1}
W20260810 07:20:11.436280 29743 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:11.431237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:51367 (local address 127.14.164.62:36167)
0810 07:20:11.431342 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:11.431344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:11.431356 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:11.431388 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:11.431389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:11.431403 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:11.431471 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:11.431836 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.432359 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.433119 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.433326 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.433385 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.436210 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:51367: BlockingRecv error: recv got EOF from 127.14.164.2:51367 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 07:20:11 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:11 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:11.590965 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.585617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:57809)
0810 07:20:11.585780 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:11.585793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:11.585865 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:11.586075 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:11.586077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:11.586286 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:11.586357 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.586359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.587082 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.587083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.587666 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.587669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.588022 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.588023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.588068 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.588568 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:11.588573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:11.590895 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
W20260810 07:20:11.591128 29743 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:11.585723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:57809 (local address 127.14.164.62:36167)
0810 07:20:11.585846 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:11.585849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:11.585864 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:11.585941 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:11.585942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:11.585957 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:11.586026 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:11.586457 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.586982 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.587756 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.587947 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.588008 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.590985 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:57809: BlockingRecv error: recv got EOF from 127.14.164.3:57809 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 10 07:20:12 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:12 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:12.297520 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:12.292833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:53361)
0810 07:20:12.292976 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:12.292989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:12.293064 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:12.293194 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:12.293195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:12.293392 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:12.293444 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:12.293447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.294115 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.294115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:12.294656 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:12.294658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.295018 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.295019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:12.295062 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.295523 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:12.295528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:12.297458 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
W20260810 07:20:12.297676 29748 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:12.292929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:53361 (local address 127.14.164.62:36167)
0810 07:20:12.293072 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:12.293074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:12.293084 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:12.293108 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:12.293109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:12.293120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:12.293180 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:12.293514 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.294044 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.294736 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.294948 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.295003 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.297582 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:53361: BlockingRecv error: recv got EOF from 127.14.164.1:53361 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
Aug 10 07:20:12 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:12 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:12.441707 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:12.436735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:42569)
0810 07:20:12.436915 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:12.436928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:12.437000 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:12.437159 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:12.437160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:12.437359 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:12.437414 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:12.437417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.438097 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.438099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:12.438659 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:12.438661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.439066 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.439067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:12.439130 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.439626 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:12.439631 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:12.441640 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
W20260810 07:20:12.441792 29748 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:12.436859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:42569 (local address 127.14.164.62:36167)
0810 07:20:12.436918 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:12.436921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:12.436977 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:20:12.437059 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:12.437060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:12.437074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:12.437150 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:12.437504 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.438011 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.438752 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.438958 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.439065 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.441741 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:42569: BlockingRecv error: recv got EOF from 127.14.164.2:42569 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 10 07:20:12 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:12 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:12.596952 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:12.591548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:59673)
0810 07:20:12.591694 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:12.591706 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:12.591801 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:12.592047 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:12.592048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:12.592282 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:12.592336 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:12.592338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.593112 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.593114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:12.593679 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:12.593681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.594128 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.594129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:12.594184 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.594761 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:12.594767 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:12.596884 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260810 07:20:12.597101 29748 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:12.591616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:59673 (local address 127.14.164.62:36167)
0810 07:20:12.591681 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:12.591683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:12.591797 (+ 114us) server_negotiation.cc:407] Connection header received
0810 07:20:12.591907 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:12.591908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:12.591922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:12.592006 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:12.592464 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.593021 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.593786 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.594001 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.594072 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.597007 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:59673: BlockingRecv error: recv got EOF from 127.14.164.3:59673 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 07:20:13 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:13 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:13.302855 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.298049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:40917)
0810 07:20:13.298235 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.298249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.298306 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.298458 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.298459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.298645 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:13.298700 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.298701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.299434 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.299435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.299970 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.299972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.300416 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.300417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.300459 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.301015 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:13.301020 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:13.302792 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
W20260810 07:20:13.303013 29754 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:13.298162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:40917 (local address 127.14.164.62:36167)
0810 07:20:13.298294 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.298301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.298311 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:13.298369 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.298370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.298381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.298441 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.298801 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.299356 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.300073 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.300335 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.300408 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.302916 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:40917: BlockingRecv error: recv got EOF from 127.14.164.1:40917 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
Aug 10 07:20:13 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:13 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:13.447288 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.442274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:56159)
0810 07:20:13.442450 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.442463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.442552 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.442710 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.442711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.442933 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:13.442984 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.442986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.443684 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.443685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.444224 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.444226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.444628 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.444629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.444697 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.445271 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:13.445277 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:13.447227 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
W20260810 07:20:13.447451 29754 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:13.442369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:56159 (local address 127.14.164.62:36167)
0810 07:20:13.442447 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.442449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.442531 (+ 82us) server_negotiation.cc:407] Connection header received
0810 07:20:13.442638 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.442639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.442651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.442695 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.443067 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.443591 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.444348 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.444550 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.444628 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.447365 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:56159: BlockingRecv error: recv got EOF from 127.14.164.2:56159 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 07:20:13 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:13 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:13.602737 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.597583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:48545)
0810 07:20:13.597783 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.597803 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.597880 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.598021 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.598022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.598256 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:13.598314 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.598316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.599049 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.599051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.599675 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.599678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.600069 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.600070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.600122 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.600603 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:13.600609 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:13.602636 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":132,"threads_started":1}
W20260810 07:20:13.602895 29754 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:13.597679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:48545 (local address 127.14.164.62:36167)
0810 07:20:13.597769 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.597771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.597880 (+ 109us) server_negotiation.cc:407] Connection header received
0810 07:20:13.597932 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.597933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.597946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.598013 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.598407 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.598950 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.599790 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.599976 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.600037 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.602764 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:48545: BlockingRecv error: recv got EOF from 127.14.164.3:48545 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 07:20:14 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:14 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:14.309067 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.303538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:60545)
0810 07:20:14.303717 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.303729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.303786 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.304005 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.304006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.304202 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.304258 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:14.304259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.305086 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.305087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:14.305739 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:14.305743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.306306 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.306307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:14.306377 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:14.306888 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:14.306895 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.309003 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
W20260810 07:20:14.309233 29760 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:14.303715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:60545 (local address 127.14.164.62:36167)
0810 07:20:14.303850 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.303852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.303860 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:20:14.303901 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.303902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.303913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.303982 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.304332 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.304982 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.305930 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.306204 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.306284 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:14.309127 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:60545: BlockingRecv error: recv got EOF from 127.14.164.1:60545 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
Aug 10 07:20:14 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:14 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:14.453394 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.447885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:54097)
0810 07:20:14.448038 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.448055 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.448109 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.448302 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.448303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.448483 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.448535 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:14.448536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.449728 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.449729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:14.450373 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:14.450376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.450752 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.450754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:14.450808 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:14.451271 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:14.451276 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.453316 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260810 07:20:14.453528 29760 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:14.447975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:54097 (local address 127.14.164.62:36167)
0810 07:20:14.448071 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.448077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.448093 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:14.448171 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.448172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.448183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.448260 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.448625 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.449639 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.450474 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.450673 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.450737 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:14.453423 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:54097: BlockingRecv error: recv got EOF from 127.14.164.2:54097 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 07:20:14 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:14 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:14.609097 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.603364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:40891)
0810 07:20:14.603654 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.603672 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.603774 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.603963 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.603965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.604246 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.604333 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:14.604336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.605102 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.605103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:14.605686 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:14.605687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.606138 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.606140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:14.606194 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:14.606859 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:14.606865 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.609028 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":94,"threads_started":1}
W20260810 07:20:14.609248 29760 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:14.603584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:40891 (local address 127.14.164.62:36167)
0810 07:20:14.603659 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.603661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.603767 (+ 106us) server_negotiation.cc:407] Connection header received
0810 07:20:14.603845 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.603846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.603865 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.603924 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.604424 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.604989 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.605796 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:14.606048 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:14.606106 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:14.609158 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:40891: BlockingRecv error: recv got EOF from 127.14.164.3:40891 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 07:20:15 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:15 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:15.321305 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.309615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:47507)
0810 07:20:15.309807 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:15.309818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:15.309910 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:15.310100 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:15.310101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:15.310300 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:15.310353 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.310355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.311297 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.311299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.312262 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.312267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.312660 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.312662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.312798 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.319050 (+ 6252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:15.319057 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:15.321205 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
W20260810 07:20:15.321429 29768 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:15.309725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:47507 (local address 127.14.164.62:36167)
0810 07:20:15.309867 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.309870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.309918 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:20:15.309993 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.309995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.310016 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.310092 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:15.310438 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.311189 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.312342 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.312570 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.312657 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.321343 (+ 8686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:47507: BlockingRecv error: recv got EOF from 127.14.164.1:47507 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
Aug 10 07:20:15 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:15 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:15.459441 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.453946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:58715)
0810 07:20:15.454155 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:15.454170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:15.454262 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:15.454440 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:15.454441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:15.454637 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:15.454691 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.454693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.455517 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.455518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.456057 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.456059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.456508 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.456509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.456577 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.457128 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:15.457133 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:15.459371 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":130,"threads_started":1}
W20260810 07:20:15.459637 29768 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:15.454059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:58715 (local address 127.14.164.62:36167)
0810 07:20:15.454140 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.454143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.454239 (+ 96us) server_negotiation.cc:407] Connection header received
0810 07:20:15.454339 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.454340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.454354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.454431 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:15.454776 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.455421 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.456147 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.456417 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.456505 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.459509 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:58715: BlockingRecv error: recv got EOF from 127.14.164.2:58715 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:20:15 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:15 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:15.614961 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.609623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:37791)
0810 07:20:15.609824 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:15.609835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:15.609940 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:15.610068 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:15.610069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:15.610250 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:15.610303 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.610305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.611225 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.611226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.611830 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.611832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.612204 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.612205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.612264 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.612940 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:15.612944 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:15.614884 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":125,"threads_started":1}
W20260810 07:20:15.615123 29768 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:15.609701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:37791 (local address 127.14.164.62:36167)
0810 07:20:15.609817 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.609819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.609938 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:20:15.609989 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.609990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.610005 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.610056 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:15.610416 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.611136 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.611925 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.612111 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.612168 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.615032 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:37791: BlockingRecv error: recv got EOF from 127.14.164.3:37791 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 07:20:16 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:16 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:16.327342 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:16.321980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:59173)
0810 07:20:16.322155 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:16.322171 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:16.322227 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:16.322457 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:16.322458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:16.322661 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:16.322713 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.322715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.323450 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.323452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.323988 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.323989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.324388 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.324389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.324458 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.325136 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:16.325142 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:16.327244 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
W20260810 07:20:16.327423 29773 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:16.322162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:59173 (local address 127.14.164.62:36167)
0810 07:20:16.322304 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:16.322306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:16.322315 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:20:16.322343 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:16.322343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:16.322355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:16.322432 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:16.322813 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.323338 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.324074 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.324278 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.324384 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.327349 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:59173: BlockingRecv error: recv got EOF from 127.14.164.1:59173 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
Aug 10 07:20:16 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:16 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:16.464824 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:16.460017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:52059)
0810 07:20:16.460176 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:16.460187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:16.460248 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:16.460388 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:16.460389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:16.460564 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:16.460618 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.460620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.461403 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.461404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.461981 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.461983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.462378 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.462379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.462431 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.462917 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:16.462922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:16.464702 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
W20260810 07:20:16.464991 29773 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:16.460104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:52059 (local address 127.14.164.62:36167)
0810 07:20:16.460157 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:16.460159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:16.460240 (+ 81us) server_negotiation.cc:407] Connection header received
0810 07:20:16.460299 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:16.460300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:16.460310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:16.460363 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:16.460685 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.461318 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.462056 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.462242 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.462328 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.464902 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:52059: BlockingRecv error: recv got EOF from 127.14.164.2:52059 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 07:20:16 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:16 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:16.620203 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:16.615434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:47075)
0810 07:20:16.615609 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:16.615620 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:16.615680 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:16.615852 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:16.615853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:16.616045 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:16.616098 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.616099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.616839 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.616840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.617368 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.617369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.617776 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.617777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.617827 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.618307 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:16.618312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:16.620139 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
W20260810 07:20:16.620433 29773 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:16.615624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:47075 (local address 127.14.164.62:36167)
0810 07:20:16.615701 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:16.615702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:16.615739 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:20:16.615767 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:16.615768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:16.615780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:16.615831 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:16.616186 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.616756 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.617479 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.617670 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.617734 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.620349 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:47075: BlockingRecv error: recv got EOF from 127.14.164.3:47075 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 07:20:17 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:17 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:17.333575 29778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:17.328017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:51637)
0810 07:20:17.328237 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:17.328250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:17.328326 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:17.328480 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:17.328481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:17.328658 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:17.328710 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:17.328731 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.329527 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.329528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:17.330069 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:17.330070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.330495 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.330496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:17.330553 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:17.331124 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:17.331130 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:17.333466 (+ 2336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":126,"threads_started":1}
W20260810 07:20:17.333647 29781 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:17.328142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:51637 (local address 127.14.164.62:36167)
0810 07:20:17.328270 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:17.328272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:17.328306 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:20:17.328399 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:17.328400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:17.328411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:17.328476 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:17.328851 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.329434 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.330158 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.330384 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.330455 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:17.333573 (+ 3118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:51637: BlockingRecv error: recv got EOF from 127.14.164.1:51637 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
Aug 10 07:20:17 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:17 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:17.470619 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:17.465480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:35317)
0810 07:20:17.465660 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:17.465671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:17.465776 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:17.465966 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:17.465967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:17.466251 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:17.466305 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:17.466308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.467058 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.467059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:17.467624 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:17.467626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.467978 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.467979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:17.468029 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:17.468525 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:17.468530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:17.470546 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
W20260810 07:20:17.470932 29781 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:17.465534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:35317 (local address 127.14.164.62:36167)
0810 07:20:17.465611 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:17.465614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:17.465767 (+ 153us) server_negotiation.cc:407] Connection header received
0810 07:20:17.465852 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:17.465853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:17.465868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:17.465928 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:17.466404 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.466946 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.467705 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.467897 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.467962 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:17.470874 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:35317: BlockingRecv error: recv got EOF from 127.14.164.2:35317 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 07:20:17 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:17 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:17.625906 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:17.620665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:33427)
0810 07:20:17.620880 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:17.620895 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:17.620978 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:17.621173 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:17.621174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:17.621396 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:17.621445 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:17.621447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.622138 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.622139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:17.622692 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:17.622694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.623065 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.623067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:17.623129 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:17.623657 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:17.623662 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:17.625834 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":127,"threads_started":1}
W20260810 07:20:17.626104 29781 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:17.620788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:33427 (local address 127.14.164.62:36167)
0810 07:20:17.620910 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:17.620913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:17.620953 (+ 40us) server_negotiation.cc:407] Connection header received
0810 07:20:17.621075 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:17.621076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:17.621097 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:17.621162 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:17.621533 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.622043 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.622782 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:17.622988 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:17.623066 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:17.626008 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:33427: BlockingRecv error: recv got EOF from 127.14.164.3:33427 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 07:20:18 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:18 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:18.339466 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.334218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:53815)
0810 07:20:18.334386 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:18.334404 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:18.334457 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:18.334646 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:18.334647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:18.334838 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:18.334893 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.334895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.335631 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.335632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.336166 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.336168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.336564 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.336565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.336632 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.337266 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:18.337272 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:18.339322 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
W20260810 07:20:18.339478 29785 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:18.334311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:53815 (local address 127.14.164.62:36167)
0810 07:20:18.334460 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:18.334462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:18.334497 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:20:18.334520 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:18.334521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:18.334531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:18.334586 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:18.335013 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.335545 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.336268 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.336472 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.336533 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.339410 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:53815: BlockingRecv error: recv got EOF from 127.14.164.1:53815 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
Aug 10 07:20:18 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:18 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:18.477205 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.471246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:36181)
0810 07:20:18.471463 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:18.471480 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:18.471601 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:18.471803 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:18.471804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:18.472020 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:18.472071 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.472073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.472963 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.472964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.473599 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.473601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.474221 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.474222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.474307 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.474902 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:18.474907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:18.477129 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
W20260810 07:20:18.477366 29785 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:18.471366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:36181 (local address 127.14.164.62:36167)
0810 07:20:18.471481 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:18.471484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:18.471582 (+ 98us) server_negotiation.cc:407] Connection header received
0810 07:20:18.471699 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:18.471700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:18.471715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:18.471786 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:18.472174 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.472855 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.473706 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.474021 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.474229 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.477274 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:36181: BlockingRecv error: recv got EOF from 127.14.164.2:36181 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 07:20:18 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:18 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:18.631945 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.626455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:57125)
0810 07:20:18.626684 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:18.626697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:18.626801 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:18.626971 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:18.626972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:18.627187 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:18.627235 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.627237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.628013 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.628014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.628699 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.628701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.629273 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.629275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.629370 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.629842 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:18.629849 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:18.631871 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":130,"threads_started":1}
W20260810 07:20:18.632052 29785 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:18.626570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:57125 (local address 127.14.164.62:36167)
0810 07:20:18.626668 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:18.626671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:18.626778 (+ 107us) server_negotiation.cc:407] Connection header received
0810 07:20:18.626862 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:18.626863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:18.626879 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:18.626960 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:18.627328 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.627849 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.628871 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.629164 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.629254 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.631955 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:57125: BlockingRecv error: recv got EOF from 127.14.164.3:57125 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 07:20:19 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:19 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:19.345721 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.340168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:60643)
0810 07:20:19.340332 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:19.340345 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:19.340411 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:19.340633 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:19.340634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:19.340972 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:19.341039 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.341042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.341838 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.341839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.342377 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.342378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.342815 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.342816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.342871 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.343497 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:19.343504 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:19.345655 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
W20260810 07:20:19.345984 29790 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:19.340235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:60643 (local address 127.14.164.62:36167)
0810 07:20:19.340448 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.340450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.340480 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:20:19.340508 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.340509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.340525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.340603 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.341133 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.341704 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.342476 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.342699 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.342762 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.345908 (+ 3146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:60643: BlockingRecv error: recv got EOF from 127.14.164.1:60643 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
Aug 10 07:20:19 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:19 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:19.482988 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.477760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:46245)
0810 07:20:19.477931 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:19.477945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:19.478020 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:19.478311 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:19.478313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:19.478508 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:19.478560 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.478562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.479331 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.479333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.479928 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.479930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.480303 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.480304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.480351 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.480882 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:19.480888 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:19.482919 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":64,"threads_started":1}
W20260810 07:20:19.483179 29790 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:19.478031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:46245 (local address 127.14.164.62:36167)
0810 07:20:19.478123 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.478125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.478137 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:19.478177 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.478178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.478190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.478254 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.478678 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.479221 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.480016 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.480211 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.480270 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.483103 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:46245: BlockingRecv error: recv got EOF from 127.14.164.2:46245 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 07:20:19 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:19 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:19.637626 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.632503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:33233)
0810 07:20:19.632673 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:19.632687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:19.632791 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:19.632934 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:19.632934 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:19.633135 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:19.633218 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.633220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.633938 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.633939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.634521 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.634524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.634978 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.634979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.635036 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.635645 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:19.635650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:19.637562 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
W20260810 07:20:19.637856 29790 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:19.632598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:33233 (local address 127.14.164.62:36167)
0810 07:20:19.632655 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.632658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.632785 (+ 127us) server_negotiation.cc:407] Connection header received
0810 07:20:19.632843 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.632843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.632856 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.632906 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.633297 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.633833 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.634637 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.634847 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.634913 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.637779 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:33233: BlockingRecv error: recv got EOF from 127.14.164.3:33233 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 07:20:20 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:20 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:20.356465 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:20.346296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:56665)
0810 07:20:20.346463 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:20.346473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:20.346523 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:20.346814 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:20.346815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:20.347002 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:20.347055 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:20.347057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.347855 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.347856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:20.348454 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:20.348456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.348939 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.348941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:20.348995 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.354008 (+ 5013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:20.354014 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:20.356356 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
W20260810 07:20:20.356755 29795 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:20.346472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:56665 (local address 127.14.164.62:36167)
0810 07:20:20.346622 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:20.346624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:20.346636 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:20.346675 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:20.346675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:20.346692 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:20.346781 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:20.347157 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.347742 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.348563 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.348824 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.348891 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.356555 (+ 7664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:56665: BlockingRecv error: recv got EOF from 127.14.164.1:56665 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
Aug 10 07:20:20 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:20 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:20.488850 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:20.483618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:59513)
0810 07:20:20.483852 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:20.483864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:20.483927 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:20.484234 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:20.484235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:20.484453 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:20.484516 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:20.484518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.485299 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.485300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:20.485841 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:20.485842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.486225 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.486226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:20.486281 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.486833 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:20.486839 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:20.488704 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":164,"threads_started":1}
W20260810 07:20:20.488977 29795 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:20.483729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:59513 (local address 127.14.164.62:36167)
0810 07:20:20.483907 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:20.483910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:20.483926 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:20.484078 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:20.484079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:20.484091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:20.484172 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:20.484617 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.485218 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.485938 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.486126 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.486188 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.488893 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:59513: BlockingRecv error: recv got EOF from 127.14.164.2:59513 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Aug 10 07:20:20 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:20 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.3@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:20.647243 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:20.638196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.3:52895)
0810 07:20:20.638458 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:20.638471 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:20.638576 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:20.638762 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:20.638764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:20.639003 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:20.639055 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:20.639057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.639772 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.639774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:20.640396 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:20.640398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.640785 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.640786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:20.640849 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.645073 (+ 4224us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:20.645079 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:20.647161 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
W20260810 07:20:20.647397 29795 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:20.638323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:52895 (local address 127.14.164.62:36167)
0810 07:20:20.638441 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:20.638443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:20.638555 (+ 112us) server_negotiation.cc:407] Connection header received
0810 07:20:20.638650 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:20.638652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:20.638673 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:20.638736 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:20.639148 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.639669 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.640479 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.640677 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.640784 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.647274 (+ 6490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.3:52895: BlockingRecv error: recv got EOF from 127.14.164.3:52895 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 07:20:21 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:21 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.1@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:21.362845 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.357204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.1:45385)
0810 07:20:21.357388 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.357404 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.357478 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.357740 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.357742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.357931 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.357987 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.357988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.358821 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.358822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.359434 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.359437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.359876 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.359877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.359954 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.360422 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.360428 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.362769 (+ 2341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
W20260810 07:20:21.363019 29804 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:21.357352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:45385 (local address 127.14.164.62:36167)
0810 07:20:21.357540 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.357542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.357554 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:21.357585 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.357585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.357598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.357683 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.358090 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.358626 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.359540 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.359795 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.359876 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.362907 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.1:45385: BlockingRecv error: recv got EOF from 127.14.164.1:45385 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
Aug 10 07:20:21 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:21 dist-test-slave-x5fp krb5kdc[28071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.164.2@KRBTEST.COM for oryx/127.14.164.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:21.494714 29805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.489569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:36167 (local address 127.14.164.2:50241)
0810 07:20:21.489755 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.489772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.489863 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.490055 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.490057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.490244 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.490302 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.490304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.490961 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.490962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.491492 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.491493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.491889 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.491890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.491938 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.492518 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.492522 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.494651 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.164.62:36167: Server oryx/127.14.164.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
W20260810 07:20:21.494930 29804 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:21.489760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:50241 (local address 127.14.164.62:36167)
0810 07:20:21.489837 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.489839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.489849 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:21.489927 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.489927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.489938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.489996 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.490388 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.490874 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.491576 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.491770 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.491826 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.494792 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.164.2:50241: BlockingRecv error: recv got EOF from 127.14.164.2:50241 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:20:21.546826 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 29035
I20260810 07:20:21.564039 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 29280
I20260810 07:20:21.576059 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 29492
I20260810 07:20:21.592257 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 28920
2026-08-10T07:20:21Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17938 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T07:20:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:21Z Disabled control of system clock
I20260810 07:20:21.628126 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:43185
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:33857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:43185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:20:21.714336 29810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:21.714570 29810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:21.714593 29810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:21.715916 29810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:21.715972 29810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:21.715986 29810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:21.715998 29810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:21.716010 29810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:21.717487 29810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:43185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:21.717790 29810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:21.718122 29810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:21.724840 29810 server_base.cc:1034] running on GCE node
W20260810 07:20:21.724793 29816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.724813 29818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.724815 29815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.725389 29810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:21.725723 29810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:21.726902 29810 hybrid_clock.cc:648] HybridClock initialized: now 1786346421726871 us; error 55 us; skew 500 ppm
I20260810 07:20:21.728516 29810 webserver.cc:460] Webserver started at http://127.14.164.62:33947/ using document root <none> and password file <none>
I20260810 07:20:21.728894 29810 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:21.728993 29810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:21.729171 29810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:21.730389 29810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "90569aea0df04e5182915464ff8ec5b4"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-x5fp"
server_key: "d319cfcbee801ebaf2528a75c42f4a1e"
server_key_iv: "38d46eb6168a023f7dc0ff15a0c74dc2"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.730792 29810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "90569aea0df04e5182915464ff8ec5b4"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-x5fp"
server_key: "d319cfcbee801ebaf2528a75c42f4a1e"
server_key_iv: "38d46eb6168a023f7dc0ff15a0c74dc2"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.732262 29810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:20:21.733366 29830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.733667 29810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:20:21.733863 29810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "90569aea0df04e5182915464ff8ec5b4"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-x5fp"
server_key: "d319cfcbee801ebaf2528a75c42f4a1e"
server_key_iv: "38d46eb6168a023f7dc0ff15a0c74dc2"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.733971 29810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:21.753337 29810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:21.753715 29810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:21.753893 29810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:21.759013 29810 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:43185
I20260810 07:20:21.759054 29908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:43185 every 8 connection(s)
I20260810 07:20:21.759482 29810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:21.760288 29909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:21.763073 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4: Bootstrap starting.
I20260810 07:20:21.763434 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 29810
I20260810 07:20:21.763554 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:20:21.763814 29909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:21.763839 14992 external_mini_cluster.cc:1428] Setting key f933e5e1c4aa3490d878a05fee056034
I20260810 07:20:21.764204 29909 log.cc:824] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:21.765152 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4: No bootstrap required, opened a new log
I20260810 07:20:21.767798 29909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90569aea0df04e5182915464ff8ec5b4" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43185 } }
I20260810 07:20:21.767959 29909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:21.767983 29909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90569aea0df04e5182915464ff8ec5b4, State: Initialized, Role: FOLLOWER
I20260810 07:20:21.768124 29909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90569aea0df04e5182915464ff8ec5b4" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43185 } }
I20260810 07:20:21.768204 29909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:21.768271 29909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:21.768322 29909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:21.769207 29909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90569aea0df04e5182915464ff8ec5b4" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43185 } }
I20260810 07:20:21.769336 29909 leader_election.cc:302] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90569aea0df04e5182915464ff8ec5b4; no voters:
I20260810 07:20:21.769553 29909 leader_election.cc:288] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:21.769620 29915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:21.770098 29915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [term 1 LEADER]: Becoming Leader. State: Replica: 90569aea0df04e5182915464ff8ec5b4, State: Running, Role: LEADER
I20260810 07:20:21.770238 29909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:21.770279 29915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90569aea0df04e5182915464ff8ec5b4" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43185 } }
I20260810 07:20:21.770741 29915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90569aea0df04e5182915464ff8ec5b4. Latest consensus state: current_term: 1 leader_uuid: "90569aea0df04e5182915464ff8ec5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90569aea0df04e5182915464ff8ec5b4" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43185 } } }
I20260810 07:20:21.770787 29917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90569aea0df04e5182915464ff8ec5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90569aea0df04e5182915464ff8ec5b4" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43185 } } }
I20260810 07:20:21.770906 29917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:21.771095 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.764494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.14.164.62:43185)
0810 07:20:21.764768 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.764777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.764929 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:20:21.765570 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.765579 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.765671 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.765832 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.766010 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.767429 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.768533 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.769198 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.769340 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.770120 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.770125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.770125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:21.770346 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.770392 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:21.770408 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.770468 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.770683 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.770769 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260810 07:20:21.771240 29915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:21.771390 29928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:21.771920 29928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:21.773890 29928 catalog_manager.cc:1347] Generated new cluster ID: aaf92237d144411280255646e9e9d526
I20260810 07:20:21.773977 29928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:21.785990 29928 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:21.786628 29928 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:21.794308 29928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90569aea0df04e5182915464ff8ec5b4: Generated new TSK 0
I20260810 07:20:21.794549 29928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:21.799244 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43185
--builtin_ntp_servers=127.14.164.20:33857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 07:20:21.888893 29950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:21.889125 29950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:21.889149 29950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:21.890365 29950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:21.890410 29950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:21.890436 29950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:20:21.891813 29950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:21.892089 29950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:21.892442 29950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:21.897992 29958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.898000 29961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.898277 29950 server_base.cc:1034] running on GCE node
W20260810 07:20:21.898078 29959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.898660 29950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:21.898972 29950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:21.900178 29950 hybrid_clock.cc:648] HybridClock initialized: now 1786346421900154 us; error 44 us; skew 500 ppm
I20260810 07:20:21.901659 29950 webserver.cc:460] Webserver started at http://127.14.164.1:41643/ using document root <none> and password file <none>
I20260810 07:20:21.901971 29950 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:21.902051 29950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:21.902220 29950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:21.903373 29950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "5dc9b9338adc4988b0c9f8d3beed071f"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-x5fp"
server_key: "7791a91650864589c803dd7955678c14"
server_key_iv: "5912a1a4289d8cebb1590c078f2a91a3"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.903780 29950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "5dc9b9338adc4988b0c9f8d3beed071f"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-x5fp"
server_key: "7791a91650864589c803dd7955678c14"
server_key_iv: "5912a1a4289d8cebb1590c078f2a91a3"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.905390 29950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:21.906359 29967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.906584 29950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:21.906700 29950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "5dc9b9338adc4988b0c9f8d3beed071f"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-x5fp"
server_key: "7791a91650864589c803dd7955678c14"
server_key_iv: "5912a1a4289d8cebb1590c078f2a91a3"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.906813 29950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:21.918638 29950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:21.919021 29950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:21.919190 29950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:21.919533 29950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:21.919942 29950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:21.919993 29950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.920019 29950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:21.920037 29950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.926577 29950 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:43093
I20260810 07:20:21.926669 30139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:43093 every 8 connection(s)
I20260810 07:20:21.927047 29950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:21.932842 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.927959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:42159 (local address 127.14.164.62:43185)
0810 07:20:21.928039 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.928041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.928313 (+ 272us) server_negotiation.cc:407] Connection header received
0810 07:20:21.928893 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.928894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.928912 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.928971 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.929789 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.930308 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.931314 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.931505 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.931562 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.932510 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.932512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.932512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:21.932517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.932531 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:21.932535 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.932587 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.932673 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.932697 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:20:21.933143 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.927877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:43185 (local address 127.14.164.1:42159)
0810 07:20:21.928141 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.928177 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.928703 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.929103 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.929107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.929117 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.929648 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.929653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.930401 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.930402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.931227 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.931229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.931589 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.931590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.931711 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.932352 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:21.932352 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.932384 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:21.932409 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:21.932597 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:21.932602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:21.932604 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:20:21.932659 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:20:21.932832 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":75,"threads_started":1}
I20260810 07:20:21.933844 30142 heartbeater.cc:347] Connected to a master server at 127.14.164.62:43185
I20260810 07:20:21.933975 30142 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:21.934182 30142 heartbeater.cc:511] Master 127.14.164.62:43185 requested a full tablet report, sending...
I20260810 07:20:21.934789 29853 ts_manager.cc:194] Registered new tserver with Master: 5dc9b9338adc4988b0c9f8d3beed071f (127.14.164.1:43093)
I20260810 07:20:21.935425 29853 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.1:42159
I20260810 07:20:21.936139 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 29950
I20260810 07:20:21.936239 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:21.936524 14992 external_mini_cluster.cc:1428] Setting key 5dbb833c7aac6fa3e229f7537f4da63e
I20260810 07:20:21.937624 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43185
--builtin_ntp_servers=127.14.164.20:33857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 07:20:22.027807 30147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.028091 30147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.028137 30147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.029371 30147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.029446 30147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.029488 30147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:20:22.030961 30147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:22.031263 30147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.031618 30147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.037652 30155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.037679 30154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.037710 30147 server_base.cc:1034] running on GCE node
W20260810 07:20:22.037670 30158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.038228 30147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.038475 30147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.039628 30147 hybrid_clock.cc:648] HybridClock initialized: now 1786346422039608 us; error 49 us; skew 500 ppm
I20260810 07:20:22.041131 30147 webserver.cc:460] Webserver started at http://127.14.164.2:42483/ using document root <none> and password file <none>
I20260810 07:20:22.041410 30147 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.041487 30147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.041649 30147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.042872 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "afff75403c9c4fe584e67900a2d215ef"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "ac068fdb8baf39955c50a8b95c3664a9"
server_key_iv: "4acd00aa41d8e26d62d808c56c211c37"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.043282 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "afff75403c9c4fe584e67900a2d215ef"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "ac068fdb8baf39955c50a8b95c3664a9"
server_key_iv: "4acd00aa41d8e26d62d808c56c211c37"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.044852 30147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:22.045943 30164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.046202 30147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.046329 30147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "afff75403c9c4fe584e67900a2d215ef"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "ac068fdb8baf39955c50a8b95c3664a9"
server_key_iv: "4acd00aa41d8e26d62d808c56c211c37"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.046430 30147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.063737 30147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.064118 30147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.064287 30147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.064550 30147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.065021 30147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.065081 30147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.065128 30147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.065169 30147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.071728 30147 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:44089
I20260810 07:20:22.071777 30336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:44089 every 8 connection(s)
I20260810 07:20:22.072188 30147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:22.074290 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 30147
I20260810 07:20:22.074424 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:22.074620 14992 external_mini_cluster.cc:1428] Setting key 862ca5f1a18513bf767a8293761c4e83
I20260810 07:20:22.075785 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43185
--builtin_ntp_servers=127.14.164.20:33857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:20:22.078428 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.073075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:56309 (local address 127.14.164.62:43185)
0810 07:20:22.073177 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.073180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.073465 (+ 285us) server_negotiation.cc:407] Connection header received
0810 07:20:22.074052 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.074053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.074068 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.074141 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.075036 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.075624 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.076757 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.076964 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.077044 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.078137 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.078138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.078139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:22.078144 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.078160 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:22.078164 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.078214 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.078325 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.078351 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 07:20:22.078701 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.073063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:43185 (local address 127.14.164.2:56309)
0810 07:20:22.073332 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.073371 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.073938 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.074229 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.074232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.074241 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.074933 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.074941 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.075722 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.075723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.076622 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.076625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.077054 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.077055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.077199 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.077961 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:22.077962 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.078005 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:22.078039 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.078232 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.078238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.078242 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.078321 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.078447 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260810 07:20:22.079303 30337 heartbeater.cc:347] Connected to a master server at 127.14.164.62:43185
I20260810 07:20:22.079414 30337 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.079627 30337 heartbeater.cc:511] Master 127.14.164.62:43185 requested a full tablet report, sending...
I20260810 07:20:22.080018 29853 ts_manager.cc:194] Registered new tserver with Master: afff75403c9c4fe584e67900a2d215ef (127.14.164.2:44089)
I20260810 07:20:22.080405 29853 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.2:56309
W20260810 07:20:22.164923 30340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.165153 30340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.165177 30340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.166402 30340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.166445 30340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.166471 30340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:20:22.167817 30340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:33857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:22.168071 30340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.168387 30340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.173911 30350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.173902 30353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.173911 30351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.174074 30340 server_base.cc:1034] running on GCE node
I20260810 07:20:22.174413 30340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.174727 30340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.175961 30340 hybrid_clock.cc:648] HybridClock initialized: now 1786346422175933 us; error 36 us; skew 500 ppm
I20260810 07:20:22.177752 30340 webserver.cc:460] Webserver started at http://127.14.164.3:45475/ using document root <none> and password file <none>
I20260810 07:20:22.178025 30340 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.178099 30340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.178246 30340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.179409 30340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "6606ac03731d4f7bb2567cbf22fe5b9e"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "0f46f0e3b4c0d015b330743d6b668b89"
server_key_iv: "0585b95bb6e5697283d31a1aa6306bed"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.179806 30340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "6606ac03731d4f7bb2567cbf22fe5b9e"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "0f46f0e3b4c0d015b330743d6b668b89"
server_key_iv: "0585b95bb6e5697283d31a1aa6306bed"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.181432 30340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:22.182221 30361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.182412 30340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:20:22.182528 30340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "6606ac03731d4f7bb2567cbf22fe5b9e"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "0f46f0e3b4c0d015b330743d6b668b89"
server_key_iv: "0585b95bb6e5697283d31a1aa6306bed"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.182610 30340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.201431 30340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.201804 30340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.201985 30340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.202257 30340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.202620 30340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.202677 30340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.202721 30340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.202764 30340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.209014 30340 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:43521
I20260810 07:20:22.209120 30532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:43521 every 8 connection(s)
I20260810 07:20:22.209515 30340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:22.213299 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 30340
I20260810 07:20:22.213410 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:22.213616 14992 external_mini_cluster.cc:1428] Setting key 256cdac99eeafa3f991a5e17414ca1a3
I20260810 07:20:22.215713 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.210384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:41325 (local address 127.14.164.62:43185)
0810 07:20:22.210449 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.210451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.210749 (+ 298us) server_negotiation.cc:407] Connection header received
0810 07:20:22.211344 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.211346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.211359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.211410 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.212168 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.212683 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.214215 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.214522 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.214596 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.215477 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.215478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.215478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:22.215482 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.215497 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:22.215500 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.215544 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.215627 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.215650 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:20:22.215926 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.210383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:43185 (local address 127.14.164.3:41325)
0810 07:20:22.210605 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.210642 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.211222 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.211488 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.211493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.211504 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.212062 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.212067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.212848 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.212849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.214130 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.214133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.214619 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.214620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.214789 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.215323 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:22.215323 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.215353 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:22.215393 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.215556 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.215561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.215563 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.215617 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.215702 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260810 07:20:22.216429 30533 heartbeater.cc:347] Connected to a master server at 127.14.164.62:43185
I20260810 07:20:22.216539 30533 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.216800 30533 heartbeater.cc:511] Master 127.14.164.62:43185 requested a full tablet report, sending...
I20260810 07:20:22.217250 29853 ts_manager.cc:194] Registered new tserver with Master: 6606ac03731d4f7bb2567cbf22fe5b9e (127.14.164.3:43521)
I20260810 07:20:22.217630 29853 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.3:41325
I20260810 07:20:22.225608 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:20:22.227530 29913 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:22.227029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46938 (local address 127.14.164.62:43185)
0810 07:20:22.227078 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.227080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.227223 (+ 143us) server_negotiation.cc:407] Connection header received
0810 07:20:22.227275 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.227276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.227286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.227326 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.227461 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46938: BlockingRecv error: recv got EOF from 127.0.0.1:46938 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:20:22.227587 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 29950
I20260810 07:20:22.240585 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 30147
I20260810 07:20:22.256474 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 30340
I20260810 07:20:22.270467 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 29810
2026-08-10T07:20:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (677 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T07:20:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:22Z Disabled control of system clock
I20260810 07:20:22.305560 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39801
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:39761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:20:22.392812 30554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.393047 30554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.393070 30554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.394227 30554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:22.394268 30554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.394281 30554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.394294 30554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:22.394306 30554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:22.395655 30554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:22.395879 30554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.396136 30554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:22.401512 30554 server_base.cc:1034] running on GCE node
W20260810 07:20:22.401556 30561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.401525 30565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.401714 30567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.402010 30554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.402330 30554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.403561 30554 hybrid_clock.cc:648] HybridClock initialized: now 1786346422403542 us; error 28 us; skew 500 ppm
I20260810 07:20:22.405159 30554 webserver.cc:460] Webserver started at http://127.14.164.62:36793/ using document root <none> and password file <none>
I20260810 07:20:22.405433 30554 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.405510 30554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.405676 30554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.406860 30554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "4f049ca9822049d99c00e932f5d2efa6"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "09eaacec30a88640d62cef5cf2c35b98"
server_key_iv: "f60f1e59af7cd750dafcbf6eace570c6"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.407261 30554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "4f049ca9822049d99c00e932f5d2efa6"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "09eaacec30a88640d62cef5cf2c35b98"
server_key_iv: "f60f1e59af7cd750dafcbf6eace570c6"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.408895 30554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:22.409673 30574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.409849 30554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:20:22.409956 30554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "4f049ca9822049d99c00e932f5d2efa6"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "09eaacec30a88640d62cef5cf2c35b98"
server_key_iv: "f60f1e59af7cd750dafcbf6eace570c6"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.410048 30554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.413550 30554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.413841 30554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.413992 30554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.417910 30554 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:39801
I20260810 07:20:22.417943 30655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:39801 every 8 connection(s)
I20260810 07:20:22.418288 30554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:22.419006 30656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:22.421607 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 30554
I20260810 07:20:22.421717 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:20:22.421926 14992 external_mini_cluster.cc:1428] Setting key 23c086c61a82ac6afc06c576d8e971b2
I20260810 07:20:22.422166 30656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6: Bootstrap starting.
I20260810 07:20:22.423043 30656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:22.423408 30656 log.cc:824] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:22.424155 30656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6: No bootstrap required, opened a new log
I20260810 07:20:22.424925 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.422550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.14.164.62:39801)
0810 07:20:22.422799 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.422808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.422843 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:20:22.423405 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.423413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.423454 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.423611 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.424385 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.424389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.424389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:22.424393 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.424438 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:22.424447 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.424543 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.424633 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.424691 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260810 07:20:22.426280 30656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f049ca9822049d99c00e932f5d2efa6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39801 } }
I20260810 07:20:22.426441 30656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:22.426493 30656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f049ca9822049d99c00e932f5d2efa6, State: Initialized, Role: FOLLOWER
I20260810 07:20:22.426625 30656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f049ca9822049d99c00e932f5d2efa6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39801 } }
I20260810 07:20:22.426709 30656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:22.426754 30656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:22.426803 30656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:22.427330 30656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f049ca9822049d99c00e932f5d2efa6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39801 } }
I20260810 07:20:22.427456 30656 leader_election.cc:302] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f049ca9822049d99c00e932f5d2efa6; no voters:
I20260810 07:20:22.427647 30656 leader_election.cc:288] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:22.427699 30663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:22.427930 30656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:22.428133 30663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [term 1 LEADER]: Becoming Leader. State: Replica: 4f049ca9822049d99c00e932f5d2efa6, State: Running, Role: LEADER
I20260810 07:20:22.428390 30663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f049ca9822049d99c00e932f5d2efa6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39801 } }
I20260810 07:20:22.429196 30664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f049ca9822049d99c00e932f5d2efa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f049ca9822049d99c00e932f5d2efa6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39801 } } }
I20260810 07:20:22.429342 30664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:22.429451 30665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f049ca9822049d99c00e932f5d2efa6. Latest consensus state: current_term: 1 leader_uuid: "4f049ca9822049d99c00e932f5d2efa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f049ca9822049d99c00e932f5d2efa6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39801 } } }
I20260810 07:20:22.429814 30665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6 [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:22.429831 30680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:22.429943 30680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:22.430070 30685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:22.430238 30685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:22.431744 30685 catalog_manager.cc:1347] Generated new cluster ID: 389cc588bd334ce895bc2a8c75cca450
I20260810 07:20:22.431804 30685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:22.444435 30685 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:22.444603 30685 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:22.453781 30685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f049ca9822049d99c00e932f5d2efa6: Generated new TSK 0
I20260810 07:20:22.453994 30685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:22.459116 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39801
--builtin_ntp_servers=127.14.164.20:39761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:20:22.554157 30689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.554425 30689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.554468 30689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.555792 30689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.555866 30689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.555917 30689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:20:22.557421 30689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:22.557729 30689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.558076 30689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.564306 30702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.564306 30697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.564298 30696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.564587 30689 server_base.cc:1034] running on GCE node
I20260810 07:20:22.564940 30689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.565256 30689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.566457 30689 hybrid_clock.cc:648] HybridClock initialized: now 1786346422566425 us; error 45 us; skew 500 ppm
I20260810 07:20:22.568137 30689 webserver.cc:460] Webserver started at http://127.14.164.1:33509/ using document root <none> and password file <none>
I20260810 07:20:22.568454 30689 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.568506 30689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.568645 30689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.569849 30689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "bf5abc89d0724c1f9f77fa5543b729b6"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "e5bc92a0e941f3d0561a1700ddccbe83"
server_key_iv: "fb5db2cc0e8755f23ba8d24cc2a84d77"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.570255 30689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "bf5abc89d0724c1f9f77fa5543b729b6"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "e5bc92a0e941f3d0561a1700ddccbe83"
server_key_iv: "fb5db2cc0e8755f23ba8d24cc2a84d77"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.571664 30689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:20:22.572521 30710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.572789 30689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.572906 30689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "bf5abc89d0724c1f9f77fa5543b729b6"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "e5bc92a0e941f3d0561a1700ddccbe83"
server_key_iv: "fb5db2cc0e8755f23ba8d24cc2a84d77"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.573001 30689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.576509 30689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.576884 30689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.577049 30689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.577289 30689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.577642 30689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.577699 30689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.577751 30689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.577780 30689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.583609 30689 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:41119
I20260810 07:20:22.583737 30899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:41119 every 8 connection(s)
I20260810 07:20:22.584071 30689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:22.585980 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 30689
I20260810 07:20:22.586095 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:22.586328 14992 external_mini_cluster.cc:1428] Setting key cf96b88ac36bd9fa7c303d2af7e694a9
I20260810 07:20:22.587421 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39801
--builtin_ntp_servers=127.14.164.20:39761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:20:22.587963 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.585133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:49413 (local address 127.14.164.62:39801)
0810 07:20:22.585202 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.585205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.585452 (+ 247us) server_negotiation.cc:407] Connection header received
0810 07:20:22.586128 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.586129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.586139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.586217 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.587677 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.587678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.587678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:22.587682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.587698 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:22.587702 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.587727 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.587875 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.587892 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:20:22.588176 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.585035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39801 (local address 127.14.164.1:49413)
0810 07:20:22.585316 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.585353 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.585977 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.586318 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.586323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.586333 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.587439 (+ 1106us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:22.587440 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.587494 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:22.587539 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.587762 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.587768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.587770 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.587886 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.587951 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260810 07:20:22.588663 30900 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39801
I20260810 07:20:22.588783 30900 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.588945 30900 heartbeater.cc:511] Master 127.14.164.62:39801 requested a full tablet report, sending...
I20260810 07:20:22.589452 30600 ts_manager.cc:194] Registered new tserver with Master: bf5abc89d0724c1f9f77fa5543b729b6 (127.14.164.1:41119)
W20260810 07:20:22.673398 30903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.673609 30903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.673632 30903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.674727 30903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.674762 30903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.674787 30903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:20:22.676195 30903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:22.676456 30903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.676838 30903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:22.682193 30903 server_base.cc:1034] running on GCE node
W20260810 07:20:22.682149 30910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.682216 30913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.682307 30916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.682557 30903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.682766 30903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.683923 30903 hybrid_clock.cc:648] HybridClock initialized: now 1786346422683893 us; error 48 us; skew 500 ppm
I20260810 07:20:22.685451 30903 webserver.cc:460] Webserver started at http://127.14.164.2:37617/ using document root <none> and password file <none>
I20260810 07:20:22.685719 30903 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.685771 30903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.685959 30903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.687135 30903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "5eaa7d23b2f44cf58cbf3727259b34ce"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "ebf726ea7d6774957ed29dcc3728da35"
server_key_iv: "03d40d1f937767d455f41503a9e8146a"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.687525 30903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "5eaa7d23b2f44cf58cbf3727259b34ce"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "ebf726ea7d6774957ed29dcc3728da35"
server_key_iv: "03d40d1f937767d455f41503a9e8146a"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.688947 30903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:22.689708 30924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.689893 30903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:20:22.690009 30903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "5eaa7d23b2f44cf58cbf3727259b34ce"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "ebf726ea7d6774957ed29dcc3728da35"
server_key_iv: "03d40d1f937767d455f41503a9e8146a"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.690091 30903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.693605 30903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.693893 30903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.694038 30903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.694243 30903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.694566 30903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.694622 30903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.694662 30903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.694692 30903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.700753 30903 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:46349
I20260810 07:20:22.700810 31098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:46349 every 8 connection(s)
I20260810 07:20:22.701159 30903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:22.703830 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 30903
I20260810 07:20:22.703963 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:22.704206 14992 external_mini_cluster.cc:1428] Setting key c1dd0cc0574d5ebf54f8b7e61d02f01f
I20260810 07:20:22.704866 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.701953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:50005 (local address 127.14.164.62:39801)
0810 07:20:22.702112 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.702115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.702346 (+ 231us) server_negotiation.cc:407] Connection header received
0810 07:20:22.702902 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.702904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.702911 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.702974 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.704524 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.704526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.704527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:22.704532 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.704551 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:22.704555 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.704613 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.704777 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.704794 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 07:20:22.705204 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.701951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39801 (local address 127.14.164.2:50005)
0810 07:20:22.702207 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.702243 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.702789 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.703058 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.703061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.703070 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.704333 (+ 1263us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:22.704334 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.704378 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:22.704422 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.704639 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.704646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.704649 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.704784 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.704853 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
I20260810 07:20:22.705437 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39801
--builtin_ntp_servers=127.14.164.20:39761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:20:22.705680 31099 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39801
I20260810 07:20:22.705786 31099 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.705971 31099 heartbeater.cc:511] Master 127.14.164.62:39801 requested a full tablet report, sending...
I20260810 07:20:22.706346 30600 ts_manager.cc:194] Registered new tserver with Master: 5eaa7d23b2f44cf58cbf3727259b34ce (127.14.164.2:46349)
W20260810 07:20:22.791033 31102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.791253 31102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.791276 31102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.792376 31102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.792409 31102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.792454 31102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:20:22.793988 31102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:39761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:22.794202 31102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.794461 31102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.799790 31114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.799804 31113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.799780 31116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.800055 31102 server_base.cc:1034] running on GCE node
I20260810 07:20:22.800326 31102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.800585 31102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.801771 31102 hybrid_clock.cc:648] HybridClock initialized: now 1786346422801768 us; error 47 us; skew 500 ppm
I20260810 07:20:22.803115 31102 webserver.cc:460] Webserver started at http://127.14.164.3:45231/ using document root <none> and password file <none>
I20260810 07:20:22.803447 31102 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.803519 31102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.803668 31102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.805171 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "058f7bbcff48463db16ee5bb78bcb6c3"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "9bd6d88a8876a9d487e2ce0f74f3581a"
server_key_iv: "d02b9707fe2f46e15604c7d18f42723f"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.805608 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "058f7bbcff48463db16ee5bb78bcb6c3"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "9bd6d88a8876a9d487e2ce0f74f3581a"
server_key_iv: "d02b9707fe2f46e15604c7d18f42723f"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.807183 31102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:22.808476 31127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.808682 31102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.808846 31102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "058f7bbcff48463db16ee5bb78bcb6c3"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-x5fp"
server_key: "9bd6d88a8876a9d487e2ce0f74f3581a"
server_key_iv: "d02b9707fe2f46e15604c7d18f42723f"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.808940 31102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.814726 31102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.815052 31102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.815186 31102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.815428 31102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.815820 31102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.815856 31102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.815879 31102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.815928 31102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.822043 31102 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:39309
I20260810 07:20:22.822085 31290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:39309 every 8 connection(s)
I20260810 07:20:22.822433 31102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:22.825558 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.823187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:37495 (local address 127.14.164.62:39801)
0810 07:20:22.823271 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.823274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.823532 (+ 258us) server_negotiation.cc:407] Connection header received
0810 07:20:22.824039 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.824040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.824045 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.824100 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.825357 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.825359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.825359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:22.825363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.825377 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:22.825381 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.825430 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.825490 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.825503 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:20:22.825794 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.823189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39801 (local address 127.14.164.3:37495)
0810 07:20:22.823403 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.823439 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.823952 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.824156 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.824159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.824168 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.825212 (+ 1044us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:22.825212 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.825242 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:22.825272 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.825430 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.825432 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.825434 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.825486 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.825536 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260810 07:20:22.826401 31291 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39801
I20260810 07:20:22.826509 31291 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.826689 31291 heartbeater.cc:511] Master 127.14.164.62:39801 requested a full tablet report, sending...
I20260810 07:20:22.827157 30600 ts_manager.cc:194] Registered new tserver with Master: 058f7bbcff48463db16ee5bb78bcb6c3 (127.14.164.3:39309)
I20260810 07:20:22.831929 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 31102
I20260810 07:20:22.832026 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:22.832278 14992 external_mini_cluster.cc:1428] Setting key b1fcf2a0a25c83feadc8e4255ed97230
I20260810 07:20:22.833623 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:20:22.835677 30660 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:22.835129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.14.164.62:39801)
0810 07:20:22.835200 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.835203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.835333 (+ 130us) server_negotiation.cc:407] Connection header received
0810 07:20:22.835378 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.835379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.835383 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.835432 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.835596 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33686: BlockingRecv error: recv got EOF from 127.0.0.1:33686 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:20:22.835713 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 30689
I20260810 07:20:22.856010 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 30903
I20260810 07:20:22.875267 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 31102
I20260810 07:20:22.891417 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 30554
2026-08-10T07:20:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (623 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:22 dist-test-slave-x5fp krb5kdc[31307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:22 dist-test-slave-x5fp krb5kdc[31307](info): set up 2 sockets
Aug 10 07:20:22 dist-test-slave-x5fp krb5kdc[31307](info): commencing operation
krb5kdc: starting...
W20260810 07:20:24.161478 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:20:24.301237 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:41475
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:41475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:24.389197 31337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:24.389429 31337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:24.389458 31337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:24.390695 31337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:24.390755 31337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:24.390771 31337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:24.390784 31337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:24.390796 31337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:24.392354 31337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:41475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:41475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:24.392654 31337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:24.392968 31337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:24.398214 31349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.398214 31347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.398232 31346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:24.398543 31337 server_base.cc:1034] running on GCE node
I20260810 07:20:24.398838 31337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:24.399219 31337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:24.400399 31337 hybrid_clock.cc:648] HybridClock initialized: now 1786346424400362 us; error 48 us; skew 500 ppm
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:24.403285 31337 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:20:24.403824 31337 webserver.cc:460] Webserver started at http://127.14.164.62:41813/ using document root <none> and password file <none>
I20260810 07:20:24.404071 31337 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:24.404141 31337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:24.404302 31337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:24.405494 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "3bac617490c24e86a4ba2d1931436eb6"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "73a0728324c828d6ef9163285206f0d6"
server_key_iv: "7b3c84f937f0d66142ee35d83e05df63"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.405887 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "3bac617490c24e86a4ba2d1931436eb6"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "73a0728324c828d6ef9163285206f0d6"
server_key_iv: "7b3c84f937f0d66142ee35d83e05df63"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.407488 31337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:24.408439 31361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.408820 31337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:24.408950 31337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "3bac617490c24e86a4ba2d1931436eb6"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "73a0728324c828d6ef9163285206f0d6"
server_key_iv: "7b3c84f937f0d66142ee35d83e05df63"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.409061 31337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:24.419894 31337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:24.422474 31337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:24.422672 31337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:24.427350 31337 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:41475
I20260810 07:20:24.427386 31439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:41475 every 8 connection(s)
I20260810 07:20:24.427802 31337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:24.428103 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 31337
I20260810 07:20:24.428231 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:20:24.428438 14992 external_mini_cluster.cc:1428] Setting key 598a58a90ee202fcc5bb4902782cdafc
I20260810 07:20:24.429333 31440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:24.432916 31440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6: Bootstrap starting.
I20260810 07:20:24.433739 31440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:24.434149 31440 log.cc:824] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:24.435112 31440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6: No bootstrap required, opened a new log
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:24.437634 31440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bac617490c24e86a4ba2d1931436eb6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41475 } }
I20260810 07:20:24.437858 31440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:24.438012 31440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bac617490c24e86a4ba2d1931436eb6, State: Initialized, Role: FOLLOWER
I20260810 07:20:24.438182 31440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bac617490c24e86a4ba2d1931436eb6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41475 } }
I20260810 07:20:24.438297 31440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:24.438346 31440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:24.438412 31440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:24.439266 31440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bac617490c24e86a4ba2d1931436eb6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41475 } }
I20260810 07:20:24.439402 31440 leader_election.cc:302] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bac617490c24e86a4ba2d1931436eb6; no voters:
I20260810 07:20:24.439625 31440 leader_election.cc:288] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:24.439692 31447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:24.439898 31447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [term 1 LEADER]: Becoming Leader. State: Replica: 3bac617490c24e86a4ba2d1931436eb6, State: Running, Role: LEADER
I20260810 07:20:24.439996 31440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:24.440011 31447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bac617490c24e86a4ba2d1931436eb6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41475 } }
I20260810 07:20:24.440487 31449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bac617490c24e86a4ba2d1931436eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bac617490c24e86a4ba2d1931436eb6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41475 } } }
I20260810 07:20:24.440478 31447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bac617490c24e86a4ba2d1931436eb6. Latest consensus state: current_term: 1 leader_uuid: "3bac617490c24e86a4ba2d1931436eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bac617490c24e86a4ba2d1931436eb6" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 41475 } } }
I20260810 07:20:24.440591 31449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:24.440598 31447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:24.440836 31452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:24.441419 31452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:24.442121 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.429077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.14.164.62:41475)
0810 07:20:24.429303 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:24.429314 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:24.429354 (+ 40us) server_negotiation.cc:407] Connection header received
0810 07:20:24.430041 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:24.430052 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:24.430122 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:24.430293 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:24.430938 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.432512 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.433459 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.434221 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.434349 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.436982 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:24.436991 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:24.436991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:24.437009 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:24.439348 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:24.439861 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:24.439865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:24.439866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:24.439983 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:24.440244 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:24.440245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:24.440246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:24.440619 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:24.440821 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:24.441505 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 07:20:24.441696 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260810 07:20:24.443336 31452 catalog_manager.cc:1347] Generated new cluster ID: 12348188f7e54f66863d97843fd9f5bc
I20260810 07:20:24.443401 31452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:24.460547 31452 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:24.461300 31452 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:24.464677 31452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6: Generated new TSK 0
I20260810 07:20:24.464972 31452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:20:24.504354 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41475
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:24.591609 31476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:24.591830 31476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:24.591856 31476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:24.592995 31476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:24.593039 31476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:24.593066 31476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:20:24.594519 31476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:24.594712 31476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:24.594907 31476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:24.600419 31487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.600453 31483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:24.600625 31476 server_base.cc:1034] running on GCE node
W20260810 07:20:24.600642 31484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:24.601001 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:24.601332 31476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:24.602531 31476 hybrid_clock.cc:648] HybridClock initialized: now 1786346424602516 us; error 28 us; skew 500 ppm
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:24.605540 31476 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:20:24.606077 31476 webserver.cc:460] Webserver started at http://127.14.164.1:33849/ using document root <none> and password file <none>
I20260810 07:20:24.606312 31476 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:24.606359 31476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:24.606539 31476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:24.607669 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "b61359436c254969b998f1ae45c2a686"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "c0805ff6e5079fb8a698f4ad22a764d5"
server_key_iv: "c5f9e3dfd8fb53a0eef0c9a0d3767fa2"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.608021 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "b61359436c254969b998f1ae45c2a686"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "c0805ff6e5079fb8a698f4ad22a764d5"
server_key_iv: "c5f9e3dfd8fb53a0eef0c9a0d3767fa2"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.609527 31476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:24.610370 31499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.610592 31476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:24.610682 31476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "b61359436c254969b998f1ae45c2a686"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "c0805ff6e5079fb8a698f4ad22a764d5"
server_key_iv: "c5f9e3dfd8fb53a0eef0c9a0d3767fa2"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.610749 31476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:24.634215 31476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:24.636638 31476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:24.636873 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:24.637126 31476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:24.637436 31476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:24.637473 31476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.637501 31476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:24.637516 31476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.644300 31476 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:38865
I20260810 07:20:24.644374 31664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:38865 every 8 connection(s)
I20260810 07:20:24.644857 31476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:24.651194 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 31476
I20260810 07:20:24.651345 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:24.651580 14992 external_mini_cluster.cc:1428] Setting key eaaa75dccf2db5928cb2de87088d4eff
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:24.655735 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.645832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:49073 (local address 127.14.164.62:41475)
0810 07:20:24.645927 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:24.645930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:24.646131 (+ 201us) server_negotiation.cc:407] Connection header received
0810 07:20:24.646620 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:24.646622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:24.646634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:24.646701 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:24.647752 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.648280 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.649355 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.649593 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.649665 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.652244 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:24.652246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:24.652246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:24.652256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:24.653946 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:24.654476 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:24.654477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:24.654478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:24.654553 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:24.654833 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:24.654834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:24.654834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:24.655038 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:24.655121 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:24.655463 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 07:20:24.655569 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:20:24.655782 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.645731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41475 (local address 127.14.164.1:49073)
0810 07:20:24.645937 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.645979 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:24.646494 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:24.646797 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:24.646801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:24.647413 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:24.647620 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.647625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.648393 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.648394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.649257 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.649259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.649684 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.649685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.649796 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.650433 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:24.650443 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:24.652125 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:24.654035 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:24.654043 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:24.654046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:24.654392 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:24.654631 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:24.654632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:24.654632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:24.654762 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:24.655141 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:24.655149 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:24.655295 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 07:20:24.655408 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:20:24.655556 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":78,"threads_started":1}
I20260810 07:20:24.656414 31665 heartbeater.cc:347] Connected to a master server at 127.14.164.62:41475
I20260810 07:20:24.656523 31665 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:24.656823 31665 heartbeater.cc:511] Master 127.14.164.62:41475 requested a full tablet report, sending...
I20260810 07:20:24.657526 31382 ts_manager.cc:194] Registered new tserver with Master: b61359436c254969b998f1ae45c2a686 (127.14.164.1:38865)
I20260810 07:20:24.658248 31382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:49073
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:20:24.683799 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41475
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:24.773434 31674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:24.773700 31674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:24.773742 31674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:24.774958 31674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:24.775036 31674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:24.775080 31674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:20:24.776639 31674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:24.777007 31674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:24.777277 31674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:24.782876 31674 server_base.cc:1034] running on GCE node
W20260810 07:20:24.782833 31683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.782894 31685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.782842 31687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:24.783288 31674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:24.783569 31674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:24.784801 31674 hybrid_clock.cc:648] HybridClock initialized: now 1786346424784782 us; error 37 us; skew 500 ppm
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:24.787689 31674 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:20:24.788226 31674 webserver.cc:460] Webserver started at http://127.14.164.2:41741/ using document root <none> and password file <none>
I20260810 07:20:24.788501 31674 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:24.788549 31674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:24.788774 31674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:24.789915 31674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "0ee91e05290b46e2a918d278f50b66fb"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "2900ea022f1a2c796c6de17c88eea10c"
server_key_iv: "796995721132ab68687521f4918cdc80"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.790632 31674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "0ee91e05290b46e2a918d278f50b66fb"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "2900ea022f1a2c796c6de17c88eea10c"
server_key_iv: "796995721132ab68687521f4918cdc80"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.792325 31674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:20:24.793311 31696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.793573 31674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:20:24.793686 31674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "0ee91e05290b46e2a918d278f50b66fb"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "2900ea022f1a2c796c6de17c88eea10c"
server_key_iv: "796995721132ab68687521f4918cdc80"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.793769 31674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:24.824115 31674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:24.826640 31674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:24.826860 31674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:24.827163 31674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:24.827518 31674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:24.827554 31674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.827580 31674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:24.827628 31674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.833847 31674 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:39105
I20260810 07:20:24.833897 31854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:39105 every 8 connection(s)
I20260810 07:20:24.834244 31674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:24.840983 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 31674
I20260810 07:20:24.841095 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:24.841362 14992 external_mini_cluster.cc:1428] Setting key 032ac028053006534647cb56a2c48b26
I20260810 07:20:24.845057 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.835099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:39369 (local address 127.14.164.62:41475)
0810 07:20:24.835186 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:24.835188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:24.835396 (+ 208us) server_negotiation.cc:407] Connection header received
0810 07:20:24.835974 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:24.835975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:24.835990 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:24.836032 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:24.837124 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.837627 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.838661 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.838862 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.838942 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.841740 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:24.841742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:24.841742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:24.841751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:24.843496 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:24.843965 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:24.843966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:24.843967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:24.844035 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:24.844320 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:24.844321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:24.844321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:24.844447 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:24.844521 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:24.844868 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 07:20:24.844959 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:20:24.845261 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.835083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41475 (local address 127.14.164.2:39369)
0810 07:20:24.835253 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.835306 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:24.835856 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:24.836119 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:24.836123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:24.836779 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:24.837015 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.837020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.837732 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.837733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.838565 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.838567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.838951 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.838952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.839088 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.839641 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:24.839650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:24.841613 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:24.843608 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:24.843618 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:24.843621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:24.843881 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:24.844116 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:24.844117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:24.844117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:24.844236 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:24.844525 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:24.844533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:24.844682 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 07:20:24.844822 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 07:20:24.845004 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260810 07:20:24.845741 31855 heartbeater.cc:347] Connected to a master server at 127.14.164.62:41475
I20260810 07:20:24.845896 31855 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:24.846143 31855 heartbeater.cc:511] Master 127.14.164.62:41475 requested a full tablet report, sending...
I20260810 07:20:24.846608 31382 ts_manager.cc:194] Registered new tserver with Master: 0ee91e05290b46e2a918d278f50b66fb (127.14.164.2:39105)
I20260810 07:20:24.847028 31382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:39369
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:20:24.872375 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41475
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:24.965044 31862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:24.965268 31862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:24.965293 31862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:24.966459 31862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:24.966503 31862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:24.966531 31862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:20:24.968040 31862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:41475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:20:24.968333 31862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:24.968588 31862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:24.974364 31862 server_base.cc:1034] running on GCE node
W20260810 07:20:24.974287 31871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.974293 31867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:24.974318 31869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:24.974682 31862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:24.975005 31862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:24.976168 31862 hybrid_clock.cc:648] HybridClock initialized: now 1786346424976138 us; error 48 us; skew 500 ppm
Aug 10 07:20:24 dist-test-slave-x5fp krb5kdc[31307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:24.979095 31862 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:20:24.979635 31862 webserver.cc:460] Webserver started at http://127.14.164.3:41421/ using document root <none> and password file <none>
I20260810 07:20:24.979919 31862 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:24.979972 31862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:24.980161 31862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:24.981406 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "c599ae02c5ae4419afd54e2a3417a0db"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "e767485e8175e95b67a1b117b5b85cf2"
server_key_iv: "7a746bbcaa4156f93710f9f790596905"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.981804 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "c599ae02c5ae4419afd54e2a3417a0db"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "e767485e8175e95b67a1b117b5b85cf2"
server_key_iv: "7a746bbcaa4156f93710f9f790596905"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.983304 31862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:24.984154 31881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:24.984395 31862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:24.984519 31862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "c599ae02c5ae4419afd54e2a3417a0db"
format_stamp: "Formatted at 2026-08-10 07:20:24 on dist-test-slave-x5fp"
server_key: "e767485e8175e95b67a1b117b5b85cf2"
server_key_iv: "7a746bbcaa4156f93710f9f790596905"
server_key_version: "encryptionkey@0"
I20260810 07:20:24.984606 31862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:25.000660 31862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:25.003010 31862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:25.003199 31862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:25.003480 31862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:25.003821 31862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:25.003880 31862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:25.003926 31862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:25.003959 31862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:25.010653 31862 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:34611
I20260810 07:20:25.010702 32068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:34611 every 8 connection(s)
I20260810 07:20:25.011055 31862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:20:25 dist-test-slave-x5fp krb5kdc[31307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346424, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:20:25.019688 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 31862
I20260810 07:20:25.019804 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:25.020085 14992 external_mini_cluster.cc:1428] Setting key cd4d6274ab5fc3714d8b9b3d9f9276d8
I20260810 07:20:25.022262 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.011890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:56873 (local address 127.14.164.62:41475)
0810 07:20:25.011963 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.011965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.012182 (+ 217us) server_negotiation.cc:407] Connection header received
0810 07:20:25.012904 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.012906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.012918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.013025 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:25.014151 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.014777 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.015894 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.016092 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.016161 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.018688 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:25.018690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:25.018690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:25.018699 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:25.020660 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:25.021217 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:25.021218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:25.021219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:25.021287 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:25.021544 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:25.021545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:25.021545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:25.021669 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:25.021762 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.022062 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.022160 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:20:25.022478 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.011862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:41475 (local address 127.14.164.3:56873)
0810 07:20:25.012039 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.012085 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:25.012704 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:25.013082 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:25.013085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:25.013820 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:25.014035 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.014041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.014901 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.014902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.015757 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.015759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.016193 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.016194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.016319 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.017013 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:25.017024 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:25.018568 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:25.020783 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:25.020791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:25.020793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:25.021122 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:25.021342 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:25.021343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:25.021344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:25.021465 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:25.021756 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:25.021764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:25.021914 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 07:20:25.022024 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 07:20:25.022194 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260810 07:20:25.022990 32069 heartbeater.cc:347] Connected to a master server at 127.14.164.62:41475
I20260810 07:20:25.023098 32069 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:25.023298 32069 heartbeater.cc:511] Master 127.14.164.62:41475 requested a full tablet report, sending...
I20260810 07:20:25.023717 31382 ts_manager.cc:194] Registered new tserver with Master: c599ae02c5ae4419afd54e2a3417a0db (127.14.164.3:34611)
I20260810 07:20:25.024103 31382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:56873
I20260810 07:20:25.032512 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:20:25.041118 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.033981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.14.164.62:41475)
0810 07:20:25.034081 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.034084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.034211 (+ 127us) server_negotiation.cc:407] Connection header received
0810 07:20:25.034329 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.034330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.034343 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.034424 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:25.035115 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.035627 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.036411 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.036621 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.036710 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.038138 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:25.038140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:25.038140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:25.038149 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:25.039720 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:25.040111 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:25.040112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:25.040112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:25.040179 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:25.040397 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:25.040398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:25.040398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:25.040568 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:25.040684 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.040888 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.040976 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:20:25.042984 31382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:25.044620 31382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:25.051999 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.047396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38865 (local address 127.0.0.1:55974)
0810 07:20:25.048114 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.048163 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:25.048263 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:25.048497 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:25.048500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:25.048503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:25.048589 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.048592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.049720 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.049720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.050848 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.050851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.051673 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.051674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.051762 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.051776 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:20:25.051858 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:20:25.051912 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260810 07:20:25.052021 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.047499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.14.164.1:38865)
0810 07:20:25.047706 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.047711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.048314 (+ 603us) server_negotiation.cc:407] Connection header received
0810 07:20:25.048368 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.048370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.048387 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.048473 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:25.048703 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.049612 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.050943 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.051577 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.051660 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.051738 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.051885 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.051929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":131,"threads_started":1}
I20260810 07:20:25.052917 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.047399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:39105 (local address 127.0.0.1:51254)
0810 07:20:25.048562 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.048571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:25.048622 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:25.048827 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:25.048828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:25.048831 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:25.048874 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.048875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.050145 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.050146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.051367 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.051370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.052599 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.052600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.052667 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.052672 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:20:25.052783 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:20:25.052826 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1152,"mutex_wait_us":32,"spinlock_wait_cycles":768,"thread_start_us":117,"threads_started":1}
I20260810 07:20:25.052933 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.047921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.14.164.2:39105)
0810 07:20:25.048093 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.048096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.048623 (+ 527us) server_negotiation.cc:407] Connection header received
0810 07:20:25.048694 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.048696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.048712 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.048813 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:25.048935 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.050022 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.051467 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.052519 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.052597 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.052654 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.052801 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.052847 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260810 07:20:25.053046 31565 tablet_service.cc:1467] Processing CreateTablet for tablet 62eab0091ae34873b2f547648298c16a (DEFAULT_TABLE table=test-table [id=a7fcbb92fa2e4415a132b559a33eb328]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:25.053305 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.047399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:34611 (local address 127.0.0.1:54444)
0810 07:20:25.048389 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.048399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:25.048479 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:25.048741 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:25.048742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:25.048745 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:25.048802 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.048804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.050145 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.050146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.051374 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.051375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.052773 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.052774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.052830 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.052834 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:25.052881 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:20:25.052912 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":103,"thread_start_us":123,"threads_started":1}
I20260810 07:20:25.053524 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.047742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.14.164.3:34611)
0810 07:20:25.047911 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.047914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.048475 (+ 561us) server_negotiation.cc:407] Connection header received
0810 07:20:25.048554 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.048557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.048572 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.048642 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:25.048886 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.050005 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.051469 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.052598 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.052663 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.052722 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.052962 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.053442 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260810 07:20:25.053519 31565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62eab0091ae34873b2f547648298c16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:25.056581 32094 tablet_bootstrap.cc:492] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: Bootstrap starting.
I20260810 07:20:25.057307 32094 tablet_bootstrap.cc:654] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:25.057399 31956 tablet_service.cc:1467] Processing CreateTablet for tablet 62eab0091ae34873b2f547648298c16a (DEFAULT_TABLE table=test-table [id=a7fcbb92fa2e4415a132b559a33eb328]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:25.057400 31762 tablet_service.cc:1467] Processing CreateTablet for tablet 62eab0091ae34873b2f547648298c16a (DEFAULT_TABLE table=test-table [id=a7fcbb92fa2e4415a132b559a33eb328]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:25.057780 31956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62eab0091ae34873b2f547648298c16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:25.057780 31762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62eab0091ae34873b2f547648298c16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:25.057982 32094 log.cc:824] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:25.058813 32094 tablet_bootstrap.cc:492] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: No bootstrap required, opened a new log
I20260810 07:20:25.058909 32094 ts_tablet_manager.cc:1397] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:25.060438 32094 raft_consensus.cc:348] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.060612 32094 raft_consensus.cc:374] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:25.060662 32094 raft_consensus.cc:729] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61359436c254969b998f1ae45c2a686, State: Initialized, Role: FOLLOWER
I20260810 07:20:25.060865 32094 consensus_queue.cc:260] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.061196 32094 ts_tablet_manager.cc:1428] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:25.061354 31665 heartbeater.cc:503] Master 127.14.164.62:41475 was elected leader, sending a full tablet report...
I20260810 07:20:25.061826 32100 tablet_bootstrap.cc:492] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: Bootstrap starting.
I20260810 07:20:25.062569 32100 tablet_bootstrap.cc:654] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:25.063186 32100 log.cc:824] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:25.063485 32101 tablet_bootstrap.cc:492] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Bootstrap starting.
I20260810 07:20:25.063989 32100 tablet_bootstrap.cc:492] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: No bootstrap required, opened a new log
I20260810 07:20:25.064052 32100 ts_tablet_manager.cc:1397] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:25.064105 32101 tablet_bootstrap.cc:654] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:25.064759 32101 log.cc:824] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:25.065527 32101 tablet_bootstrap.cc:492] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: No bootstrap required, opened a new log
I20260810 07:20:25.065587 32101 ts_tablet_manager.cc:1397] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:25.065893 32100 raft_consensus.cc:348] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.066056 32100 raft_consensus.cc:374] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:25.066109 32100 raft_consensus.cc:729] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee91e05290b46e2a918d278f50b66fb, State: Initialized, Role: FOLLOWER
I20260810 07:20:25.066243 32100 consensus_queue.cc:260] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.066519 32100 ts_tablet_manager.cc:1428] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:25.066597 31855 heartbeater.cc:503] Master 127.14.164.62:41475 was elected leader, sending a full tablet report...
I20260810 07:20:25.067618 32101 raft_consensus.cc:348] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.067800 32101 raft_consensus.cc:374] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:25.067828 32101 raft_consensus.cc:729] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c599ae02c5ae4419afd54e2a3417a0db, State: Initialized, Role: FOLLOWER
I20260810 07:20:25.067947 32101 consensus_queue.cc:260] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.068270 32101 ts_tablet_manager.cc:1428] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:25.068301 32069 heartbeater.cc:503] Master 127.14.164.62:41475 was elected leader, sending a full tablet report...
W20260810 07:20:25.084990 31857 tablet.cc:2369] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:20:25.145777 31666 tablet.cc:2369] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:25.169587 32107 raft_consensus.cc:482] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:25.169782 32107 raft_consensus.cc:504] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.170270 32107 leader_election.cc:288] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b61359436c254969b998f1ae45c2a686 (127.14.164.1:38865), 0ee91e05290b46e2a918d278f50b66fb (127.14.164.2:39105)
I20260810 07:20:25.173324 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.170443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38865 (local address 127.14.164.3:38449)
0810 07:20:25.170546 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.170561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:25.170616 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:25.170873 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:25.170875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:25.170877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:25.170966 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.170968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.171626 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.171627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.172384 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.172386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.173017 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.173018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.173110 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.173114 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:25.173190 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:20:25.173247 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":3}
I20260810 07:20:25.173363 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.170678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:38449 (local address 127.14.164.1:38865)
0810 07:20:25.170755 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.170757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.170764 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:20:25.170790 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.170790 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.170798 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.170866 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:25.171050 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.171539 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.172500 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.172932 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.173021 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.173031 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.173215 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.173276 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:20:25.173704 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.170597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:48267 (local address 127.14.164.2:39105)
0810 07:20:25.170678 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.170680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.170806 (+ 126us) server_negotiation.cc:407] Connection header received
0810 07:20:25.170901 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.170902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.170923 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.171004 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:25.171181 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.171649 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.172668 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.173221 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.173304 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.173313 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.173559 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.173616 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:20:25.173684 31596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eab0091ae34873b2f547648298c16a" candidate_uuid: "c599ae02c5ae4419afd54e2a3417a0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61359436c254969b998f1ae45c2a686" is_pre_election: true
I20260810 07:20:25.173869 31596 raft_consensus.cc:2454] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c599ae02c5ae4419afd54e2a3417a0db in term 0.
I20260810 07:20:25.173945 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.170527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:39105 (local address 127.14.164.3:48267)
0810 07:20:25.170725 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.170735 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:25.170824 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:25.171011 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:25.171012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:25.171014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:25.171091 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.171093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.171756 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.171757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.172564 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:25.172566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.173353 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.173354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:25.173440 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.173444 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:25.173815 (+ 371us) client_negotiation.cc:240] Negotiation successful
0810 07:20:25.173867 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":10,"thread_start_us":111,"threads_started":1}
I20260810 07:20:25.174125 31885 leader_election.cc:302] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b61359436c254969b998f1ae45c2a686, c599ae02c5ae4419afd54e2a3417a0db; no voters:
I20260810 07:20:25.174299 32107 raft_consensus.cc:2781] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:25.174365 32107 raft_consensus.cc:482] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:25.174391 32107 raft_consensus.cc:3037] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:25.174403 31790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eab0091ae34873b2f547648298c16a" candidate_uuid: "c599ae02c5ae4419afd54e2a3417a0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee91e05290b46e2a918d278f50b66fb" is_pre_election: true
I20260810 07:20:25.174558 31790 raft_consensus.cc:2454] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c599ae02c5ae4419afd54e2a3417a0db in term 0.
I20260810 07:20:25.175242 32107 raft_consensus.cc:504] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.175390 32107 leader_election.cc:288] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [CANDIDATE]: Term 1 election: Requested vote from peers b61359436c254969b998f1ae45c2a686 (127.14.164.1:38865), 0ee91e05290b46e2a918d278f50b66fb (127.14.164.2:39105)
I20260810 07:20:25.175570 31790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eab0091ae34873b2f547648298c16a" candidate_uuid: "c599ae02c5ae4419afd54e2a3417a0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee91e05290b46e2a918d278f50b66fb"
I20260810 07:20:25.175593 31596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eab0091ae34873b2f547648298c16a" candidate_uuid: "c599ae02c5ae4419afd54e2a3417a0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61359436c254969b998f1ae45c2a686"
I20260810 07:20:25.175645 31790 raft_consensus.cc:3037] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:25.175738 31596 raft_consensus.cc:3037] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:25.176450 31790 raft_consensus.cc:2454] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c599ae02c5ae4419afd54e2a3417a0db in term 1.
I20260810 07:20:25.176586 31596 raft_consensus.cc:2454] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c599ae02c5ae4419afd54e2a3417a0db in term 1.
I20260810 07:20:25.176640 31882 leader_election.cc:302] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee91e05290b46e2a918d278f50b66fb, c599ae02c5ae4419afd54e2a3417a0db; no voters:
I20260810 07:20:25.176759 32107 raft_consensus.cc:2781] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:25.176931 32107 raft_consensus.cc:686] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 1 LEADER]: Becoming Leader. State: Replica: c599ae02c5ae4419afd54e2a3417a0db, State: Running, Role: LEADER
I20260810 07:20:25.177044 32107 consensus_queue.cc:237] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } }
I20260810 07:20:25.177912 31381 catalog_manager.cc:5495] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db reported cstate change: term changed from 0 to 1, leader changed from <none> to c599ae02c5ae4419afd54e2a3417a0db (127.14.164.3). New cstate: current_term: 1 leader_uuid: "c599ae02c5ae4419afd54e2a3417a0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c599ae02c5ae4419afd54e2a3417a0db" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:25.208250 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.205516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.14.164.3:34611)
0810 07:20:25.205589 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:25.205591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:25.205598 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:20:25.205651 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:25.205652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:25.205672 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:25.205797 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:25.205968 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.206485 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.207343 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:25.207562 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:25.207653 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:25.207839 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:25.207960 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:25.208053 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:25.208094 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:20:25.208119 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:20:25.210358 31790 raft_consensus.cc:1261] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 1 FOLLOWER]: Refusing update from remote peer c599ae02c5ae4419afd54e2a3417a0db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:25.210366 31596 raft_consensus.cc:1261] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686 [term 1 FOLLOWER]: Refusing update from remote peer c599ae02c5ae4419afd54e2a3417a0db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:25.210830 32110 consensus_queue.cc:1035] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee91e05290b46e2a918d278f50b66fb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:25.211040 32107 consensus_queue.cc:1035] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "b61359436c254969b998f1ae45c2a686" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:25.213905 32118 mvcc.cc:205] Tried to move back new op lower bound from 7316874957658382336 to 7316874957525794816. Current Snapshot: MvccSnapshot[applied={T|T < 7316874957658382336}]
I20260810 07:20:25.216861 31382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:25.216993 31382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:25.217922 31382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bac617490c24e86a4ba2d1931436eb6: Sending DeleteTablet for 3 replicas of tablet 62eab0091ae34873b2f547648298c16a
I20260810 07:20:25.218330 31956 tablet_service.cc:1514] Processing DeleteTablet for tablet 62eab0091ae34873b2f547648298c16a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:25 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:54444
I20260810 07:20:25.218350 31762 tablet_service.cc:1514] Processing DeleteTablet for tablet 62eab0091ae34873b2f547648298c16a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:25 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:51254
I20260810 07:20:25.218427 31565 tablet_service.cc:1514] Processing DeleteTablet for tablet 62eab0091ae34873b2f547648298c16a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:25 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:55974
I20260810 07:20:25.218780 32139 tablet_replica.cc:330] T 62eab0091ae34873b2f547648298c16a P b61359436c254969b998f1ae45c2a686: stopping tablet replica
I20260810 07:20:25.218845 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 31476
I20260810 07:20:25.218756 32134 tablet_replica.cc:330] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: stopping tablet replica
I20260810 07:20:25.246462 32134 raft_consensus.cc:2229] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:25.246696 32134 raft_consensus.cc:2258] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:25.247249 32134 ts_tablet_manager.cc:1905] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:25.249596 32134 ts_tablet_manager.cc:1918] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:25.249699 32134 log.cc:1196] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/62eab0091ae34873b2f547648298c16a
I20260810 07:20:25.250020 32134 ts_tablet_manager.cc:1939] T 62eab0091ae34873b2f547648298c16a P c599ae02c5ae4419afd54e2a3417a0db: Deleting consensus metadata
I20260810 07:20:25.250535 32137 tablet_replica.cc:330] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb: stopping tablet replica
I20260810 07:20:25.250572 31365 catalog_manager.cc:4841] TS c599ae02c5ae4419afd54e2a3417a0db (127.14.164.3:34611): tablet 62eab0091ae34873b2f547648298c16a (table test-table [id=a7fcbb92fa2e4415a132b559a33eb328]) successfully deleted
I20260810 07:20:25.250718 32137 raft_consensus.cc:2229] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:20:25.250799 31364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:38865 (error 108)
I20260810 07:20:25.250850 32137 raft_consensus.cc:2258] T 62eab0091ae34873b2f547648298c16a P 0ee91e05290b46e2a918d278f50b66fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:25.251210 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 31674
I20260810 07:20:25.261821 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.261591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:38865 (local address 127.0.0.1:55976)
0810 07:20:25.261676 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.261753 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:38865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 07:20:25.262184 31364 catalog_manager.cc:4584] TS b61359436c254969b998f1ae45c2a686 (127.14.164.1:38865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62eab0091ae34873b2f547648298c16a: Network error: Client connection negotiation failed: client connection to 127.14.164.1:38865: connect: Connection refused (error 111)
I20260810 07:20:25.275264 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 31862
I20260810 07:20:25.275456 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.275257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:39105 (local address 127.0.0.1:51270)
0810 07:20:25.275353 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.275404 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.2:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 07:20:25.296864 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 31337
2026-08-10T07:20:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2405 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:25 dist-test-slave-x5fp krb5kdc[32144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:25 dist-test-slave-x5fp krb5kdc[32144](info): set up 2 sockets
Aug 10 07:20:25 dist-test-slave-x5fp krb5kdc[32144](info): commencing operation
krb5kdc: starting...
W20260810 07:20:26.650714 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:20:26 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/rangeradmin_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/rangeradmin_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/rangerlookup_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/rangerlookup_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
I20260810 07:20:26.840808 14992 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMAuLWb/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/postgres -l logfile start
2026-08-10 07:20:29.753 UTC [32185] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:20:29.753 UTC [32185] LOG: listening on IPv4 address "127.14.164.20", port 40905
2026-08-10 07:20:29.758 UTC [32185] LOG: listening on Unix socket "/tmp/.s.PGSQL.40905"
2026-08-10 07:20:29.797 UTC [32189] LOG: database system was shut down at 2026-08-10 07:20:28 UTC
2026-08-10 07:20:29.802 UTC [32185] LOG: database system is ready to accept connections
I20260810 07:20:30.997849 14992 mini_postgres.cc:96] Postgres bound to 40905
2026-08-10 07:20:31.007 UTC [32201] FATAL: database "slave" does not exist
127.14.164.20:40905 - accepting connections
I20260810 07:20:31.008445 14992 mini_ranger.cc:161] Starting Ranger...
I20260810 07:20:31.025736 14992 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:20:31.695900 14992 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:20:31.695988 14992 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-admin
2026-08-10 07:20:32,444 [I] DB FLAVOR :POSTGRES
2026-08-10 07:20:32,444 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:20:32,445 [I] Checking connection..
2026-08-10 07:20:32,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:20:32,885 [I] Checking connection passed.
2026-08-10 07:20:32,886 [I] --------- Verifying version history table ---------
2026-08-10 07:20:32,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:20:33,153 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:20:33,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:20:33,456 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:20:33,456 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:20:33,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:20:33.725 UTC [32305] WARNING: there is no transaction in progress
2026-08-10 07:20:33,735 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:20:33,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:20:34,015 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:20:34,015 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:20:34,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:20:34,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:20:34,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x5fp.c.gcp-upstream.internal','N') ;"
2026-08-10 07:20:34,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:20:35,122 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:20:35,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:20:35,415 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:20:35,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:20:35,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:20:35,957 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:20:35,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:20:37.032 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.050 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.068 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.087 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.106 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.126 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.200 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.209 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.226 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.238 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.250 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.265 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.277 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.285 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.293 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:37.301 UTC [32464] WARNING: there is no transaction in progress
2026-08-10 07:20:38,099 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:20:38,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:20:38,370 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:20:38,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:20:38,646 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:20:38,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:20:38,919 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:20:38,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:20:39,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:20:39,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x5fp.c.gcp-upstream.internal';"
2026-08-10 07:20:39,784 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:20:39,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:20:40,036 [I] DB_PATCHES have already been applied
I20260810 07:20:40.042870 14992 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMAuLWb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:20:40.042948 14992 mini_ranger.cc:193] Using host: 127.14.164.20
I20260810 07:20:40.047148 14992 mini_ranger.cc:227] Ranger admin URL: http://127.14.164.20:55603
Aug 10, 2026 7:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:20:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:20:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:20:41.444420 14992 mini_ranger.cc:160] Time spent starting Ranger: real 10.436s user 0.000s sys 0.012s
Aug 10, 2026 7:21:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:21:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 10, 2026 7:21:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:21:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:21:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:21:04.656644 14992 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/rangerkms_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/rangerkms_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.164.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.164.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:21:04 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346464, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:21:04.755511 14992 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:21:04.767249 14992 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:21:05.786478 14992 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:21:05.786580 14992 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms
I20260810 07:21:05.786603 14992 mini_ranger_kms.cc:227] Using postgres at 127.14.164.20:40905
2026-08-10 07:21:06,651 [I] DB FLAVOR :POSTGRES
2026-08-10 07:21:06,651 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:21:06,651 [I] Checking connection
2026-08-10 07:21:06,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:21:07,001 [I] connection success
2026-08-10 07:21:07,001 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:21:07,001 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:21:07,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:21:07,279 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:21:07,280 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:21:07,280 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:21:07,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:40905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:21:07,600 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:21:08.163066 14992 mini_ranger_kms.cc:310] Created kms service
I20260810 07:21:08.354635 14992 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:21:08.404588 14992 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:21:08 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346464, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:21:08.888034 14992 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:21:08.888173 14992 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:21:08.888197 14992 mini_ranger_kms.cc:240] Using host: 127.14.164.20
I20260810 07:21:08.893178 14992 mini_ranger_kms.cc:276] Ranger KMS PID: 32725
I20260810 07:21:08.893345 14992 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.164.20:45963
Aug 10, 2026 7:21:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:21:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:21:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:21:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:21:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:21:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:21:11 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346471, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.164.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:11.734455 14992 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.979s user 0.012s sys 0.558s
I20260810 07:21:11.734575 14992 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:21:11.734648 14992 mini_ranger_kms.cc:401] 127.14.164.20:45963/kms/v1/keys
Aug 10 07:21:12 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346471, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.164.20@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
Aug 10 07:21:12 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346471, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.164.20@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
Aug 10 07:21:15 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:21:15.260813 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40083
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:40083
--ranger_config_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:45963/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:21:15.378054 332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:15.378305 332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:15.378333 332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:15.379746 332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:21:15.379801 332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:15.379818 332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:15.379832 332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:21:15.379848 332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:21:15.381947 332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.164.20:45963/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:40083
--ranger_config_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:21:15.382270 332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:15.382552 332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:15.389353 343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:15.389429 332 server_base.cc:1034] running on GCE node
W20260810 07:21:15.389556 345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:15.389778 342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:15.390172 332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:15.390549 332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:15.391757 332 hybrid_clock.cc:648] HybridClock initialized: now 1786346475391727 us; error 45 us; skew 500 ppm
Aug 10 07:21:15 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346475, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:15.395655 332 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:21:15.396306 332 webserver.cc:460] Webserver started at http://127.14.164.62:43809/ using document root <none> and password file <none>
I20260810 07:21:15.396600 332 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:15.396646 332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:15.396829 332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:15 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346475, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:21:15.453071 332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "d0cb071ade0b4baf9c28efc083b6ddff"
format_stamp: "Formatted at 2026-08-10 07:21:15 on dist-test-slave-x5fp"
server_key: "2b5009c1cd2d287f071f9d33fb09d200"
server_key_iv: "33c2f343a8d919369440da0eebb8f372"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:15.453608 332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "d0cb071ade0b4baf9c28efc083b6ddff"
format_stamp: "Formatted at 2026-08-10 07:21:15 on dist-test-slave-x5fp"
server_key: "2b5009c1cd2d287f071f9d33fb09d200"
server_key_iv: "33c2f343a8d919369440da0eebb8f372"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:15.455544 332 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:21:15.470289 359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:15.470548 332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:21:15.470671 332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "d0cb071ade0b4baf9c28efc083b6ddff"
format_stamp: "Formatted at 2026-08-10 07:21:15 on dist-test-slave-x5fp"
server_key: "2b5009c1cd2d287f071f9d33fb09d200"
server_key_iv: "33c2f343a8d919369440da0eebb8f372"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:15.470811 332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:15.480113 332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:15.482573 332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:15.482800 332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:15.490716 332 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:40083
I20260810 07:21:15.490908 437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:40083 every 8 connection(s)
I20260810 07:21:15.491217 332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:21:15.492238 438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:15.495353 438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff: Bootstrap starting.
I20260810 07:21:15.496173 438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:15.496778 438 log.cc:824] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:15.497608 438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff: No bootstrap required, opened a new log
I20260810 07:21:15.499892 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 332
I20260810 07:21:15.500027 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:21:15.500248 14992 external_mini_cluster.cc:1428] Setting key 017a23ebe70702552d35b719d123f82a
I20260810 07:21:15.500651 438 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40083 } }
I20260810 07:21:15.500949 438 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:15.501039 438 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0cb071ade0b4baf9c28efc083b6ddff, State: Initialized, Role: FOLLOWER
I20260810 07:21:15.501206 438 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40083 } }
I20260810 07:21:15.501307 438 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:21:15.501338 438 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:21:15.501396 438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:15.502179 438 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40083 } }
I20260810 07:21:15.502314 438 leader_election.cc:302] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0cb071ade0b4baf9c28efc083b6ddff; no voters:
I20260810 07:21:15.502470 438 leader_election.cc:288] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:21:15.502691 450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:21:15.502738 438 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:21:15.502965 450 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [term 1 LEADER]: Becoming Leader. State: Replica: d0cb071ade0b4baf9c28efc083b6ddff, State: Running, Role: LEADER
I20260810 07:21:15.503136 450 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40083 } }
I20260810 07:21:15.503665 449 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40083 } } }
I20260810 07:21:15.503762 449 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [sys.catalog]: This master's current role is: LEADER
I20260810 07:21:15.504390 452 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0cb071ade0b4baf9c28efc083b6ddff. Latest consensus state: current_term: 1 leader_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cb071ade0b4baf9c28efc083b6ddff" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40083 } } }
I20260810 07:21:15.504477 452 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff [sys.catalog]: This master's current role is: LEADER
I20260810 07:21:15.504833 462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:21:15.504992 438 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:21:15.505537 462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:21:15 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:21:15.516840 462 catalog_manager.cc:1347] Generated new cluster ID: d01b7669a223498aa86373fe95a1479a
I20260810 07:21:15.517019 462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:21:15.518877 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:15.500951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.14.164.62:40083)
0810 07:21:15.501236 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:15.501246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:15.501283 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:21:15.501409 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:15.501417 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:15.501479 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:15.501637 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:15.502312 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:15.503952 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:15.505420 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:15.509831 (+ 4411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:15.510216 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:15.513615 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:15.513620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:15.513620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:15.513634 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:15.516582 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:15.517161 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:15.517165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:15.517165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:15.517349 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:15.517634 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:15.517635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:15.517635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:15.518033 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:15.518217 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:15.518370 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 07:21:15.518520 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":187,"threads_started":1}
I20260810 07:21:15.528185 462 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:21:15.528833 462 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:21:15.541352 462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff: Generated new TSK 0
I20260810 07:21:15.541599 462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:21:16.380686 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:45963/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40083
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:21:16.497793 522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:16.498047 522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:16.498092 522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:16.499452 522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:16.499529 522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:16.499575 522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:21:16.501237 522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.164.20:45963/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:21:16.501511 522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:16.501833 522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:16.508667 533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:16.508670 529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:16.509055 531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:16.509495 522 server_base.cc:1034] running on GCE node
I20260810 07:21:16.509747 522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:16.510012 522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:16.511201 522 hybrid_clock.cc:648] HybridClock initialized: now 1786346476511192 us; error 39 us; skew 500 ppm
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:16.515019 522 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:21:16.515590 522 webserver.cc:460] Webserver started at http://127.14.164.1:41795/ using document root <none> and password file <none>
I20260810 07:21:16.515883 522 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:16.515966 522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:16.516157 522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:21:16.531663 522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2"
format_stamp: "Formatted at 2026-08-10 07:21:16 on dist-test-slave-x5fp"
server_key: "de9812be30e6463a831f102b0fa8237d"
server_key_iv: "e1f267f2b4a10a7b38ccd0365171ee16"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:16.532315 522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2"
format_stamp: "Formatted at 2026-08-10 07:21:16 on dist-test-slave-x5fp"
server_key: "de9812be30e6463a831f102b0fa8237d"
server_key_iv: "e1f267f2b4a10a7b38ccd0365171ee16"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:16.534863 522 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 07:21:16.549160 540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:16.549376 522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:21:16.549511 522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2"
format_stamp: "Formatted at 2026-08-10 07:21:16 on dist-test-slave-x5fp"
server_key: "de9812be30e6463a831f102b0fa8237d"
server_key_iv: "e1f267f2b4a10a7b38ccd0365171ee16"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:16.549643 522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:16.563077 522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:16.565790 522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:16.565979 522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:16.566370 522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:21:16.566778 522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:21:16.566833 522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:16.566905 522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:21:16.566951 522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:16.580611 522 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:32941
I20260810 07:21:16.580938 714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:32941 every 8 connection(s)
I20260810 07:21:16.581221 522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:21:16.589699 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 522
I20260810 07:21:16.589879 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:21:16.590099 14992 external_mini_cluster.cc:1428] Setting key f4b238941acc6c10a9353a0125820957
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:21:16.596056 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:16.582290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37685 (local address 127.14.164.62:40083)
0810 07:21:16.582481 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:16.582484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:16.582915 (+ 431us) server_negotiation.cc:407] Connection header received
0810 07:21:16.583971 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:16.583972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:16.583987 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:16.584061 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:16.585256 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.586181 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.588304 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.588527 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.588596 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:16.591868 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:16.591870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:16.591870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:16.591880 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:16.594179 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:16.594670 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:16.594671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:16.594672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:16.594749 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:16.595098 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:16.595099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:16.595100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:16.595321 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:16.595420 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:16.595822 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 07:21:16.595927 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260810 07:21:16.596359 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:16.582308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40083 (local address 127.14.164.1:37685)
0810 07:21:16.582683 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:16.582741 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:16.583831 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:16.584169 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:16.584174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:16.584566 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:21:16.585090 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:16.585098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.586481 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.586482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:16.588176 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:16.588180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.588640 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.588641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:16.588912 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:16.589701 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:21:16.589713 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:21:16.591750 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:21:16.594308 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:16.594318 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:16.594320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:16.594581 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:16.594829 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:16.594830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:16.594831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:16.595012 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:16.595425 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:21:16.595432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:21:16.595629 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 07:21:16.595773 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:21:16.595979 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":161,"threads_started":1}
I20260810 07:21:16.597280 715 heartbeater.cc:347] Connected to a master server at 127.14.164.62:40083
I20260810 07:21:16.597496 715 heartbeater.cc:464] Registering TS with master...
I20260810 07:21:16.597847 715 heartbeater.cc:511] Master 127.14.164.62:40083 requested a full tablet report, sending...
I20260810 07:21:16.598634 386 ts_manager.cc:194] Registered new tserver with Master: d9fe940bc2754e4a80f0f3630fb6e3a2 (127.14.164.1:32941)
I20260810 07:21:16.599872 386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:37685
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:21:16.624495 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:45963/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40083
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:21:16.771271 727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:16.771667 727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:16.771775 727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:16.773756 727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:16.773901 727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:16.773998 727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:21:16.776829 727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.164.20:45963/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:21:16.777354 727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:16.777711 727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:16.786069 740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:16.786157 738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:16.786516 727 server_base.cc:1034] running on GCE node
W20260810 07:21:16.786803 742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:16.787099 727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:16.787395 727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:16.788575 727 hybrid_clock.cc:648] HybridClock initialized: now 1786346476788547 us; error 55 us; skew 500 ppm
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:16.793148 727 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:21:16.793782 727 webserver.cc:460] Webserver started at http://127.14.164.2:42063/ using document root <none> and password file <none>
I20260810 07:21:16.794080 727 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:16.794150 727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:16.794364 727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:21:16.810914 727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "578267532eb747c499ece61881bdedf6"
format_stamp: "Formatted at 2026-08-10 07:21:16 on dist-test-slave-x5fp"
server_key: "a432d0a91798beafb8e317ab19e9eb27"
server_key_iv: "f9454a8b54d8d666b8be0193c1513fee"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:16.812065 727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "578267532eb747c499ece61881bdedf6"
format_stamp: "Formatted at 2026-08-10 07:21:16 on dist-test-slave-x5fp"
server_key: "a432d0a91798beafb8e317ab19e9eb27"
server_key_iv: "f9454a8b54d8d666b8be0193c1513fee"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:16.820581 727 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260810 07:21:16.835350 755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:16.835642 727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:21:16.835824 727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "578267532eb747c499ece61881bdedf6"
format_stamp: "Formatted at 2026-08-10 07:21:16 on dist-test-slave-x5fp"
server_key: "a432d0a91798beafb8e317ab19e9eb27"
server_key_iv: "f9454a8b54d8d666b8be0193c1513fee"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:16.836028 727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:16.846278 727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:16.848773 727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:16.848955 727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:16.849242 727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:21:16.849654 727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:21:16.849711 727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:16.849781 727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:21:16.849831 727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:16.865237 727 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:40001
I20260810 07:21:16.865918 727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:21:16.871984 928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:40001 every 8 connection(s)
I20260810 07:21:16.875164 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 727
I20260810 07:21:16.875289 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:21:16.875497 14992 external_mini_cluster.cc:1428] Setting key 8e18fa833db2948592c93d8133c3c10d
Aug 10 07:21:16 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:21:16.886891 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:16.869203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:50053 (local address 127.14.164.62:40083)
0810 07:21:16.869298 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:16.869301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:16.869450 (+ 149us) server_negotiation.cc:407] Connection header received
0810 07:21:16.870461 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:16.870462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:16.870483 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:16.870577 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:16.871606 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.872479 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.874406 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.874847 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.874899 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:16.882154 (+ 7255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:16.882156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:16.882157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:16.882179 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:16.884356 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:16.885176 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:16.885178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:16.885178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:16.885322 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:16.885736 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:16.885737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:16.885738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:16.886052 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:16.886136 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:16.886650 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 07:21:16.886775 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:21:16.887574 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:16.868846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40083 (local address 127.14.164.2:50053)
0810 07:21:16.869310 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:16.869361 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:16.870326 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:16.870683 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:16.870687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:16.871037 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:21:16.871465 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:16.871473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.872624 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.872636 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:16.874270 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:16.874282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:16.875136 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:16.875137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:16.875356 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:16.876448 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:21:16.876461 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:21:16.881992 (+ 5531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:21:16.884511 (+ 2519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:16.884528 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:16.884532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:16.885026 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:16.885416 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:16.885416 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:16.885417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:16.885644 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:16.886197 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:21:16.886206 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:21:16.886443 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 07:21:16.887029 (+ 586us) client_negotiation.cc:240] Negotiation successful
0810 07:21:16.887283 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":130,"threads_started":1}
I20260810 07:21:16.888509 929 heartbeater.cc:347] Connected to a master server at 127.14.164.62:40083
I20260810 07:21:16.888677 929 heartbeater.cc:464] Registering TS with master...
I20260810 07:21:16.888955 929 heartbeater.cc:511] Master 127.14.164.62:40083 requested a full tablet report, sending...
I20260810 07:21:16.889463 386 ts_manager.cc:194] Registered new tserver with Master: 578267532eb747c499ece61881bdedf6 (127.14.164.2:40001)
I20260810 07:21:16.890071 386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:50053
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:21:16.924459 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:45963/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40083
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:21:17.083052 938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:17.083485 938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:17.083546 938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:17.085623 938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:17.085731 938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:17.085811 938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:21:17.088414 938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.164.20:45963/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:40083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:21:17.088867 938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:17.089180 938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:17.096848 945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:17.097157 946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:17.096848 948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:17.098287 938 server_base.cc:1034] running on GCE node
I20260810 07:21:17.098547 938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:17.098896 938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:17.100036 938 hybrid_clock.cc:648] HybridClock initialized: now 1786346477100011 us; error 41 us; skew 500 ppm
Aug 10 07:21:17 dist-test-slave-x5fp krb5kdc[32144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346477, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:17.104394 938 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:21:17.105191 938 webserver.cc:460] Webserver started at http://127.14.164.3:34739/ using document root <none> and password file <none>
I20260810 07:21:17.105495 938 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:17.105572 938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:17.105790 938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:17 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346477, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:21:17.120451 938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "d37a92c092954acfbf78271d06972e68"
format_stamp: "Formatted at 2026-08-10 07:21:17 on dist-test-slave-x5fp"
server_key: "6a1b90ef14592b77f0331c02c62ba31e"
server_key_iv: "5706421821b8f6e96d72096abd8b262b"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:17.121102 938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "d37a92c092954acfbf78271d06972e68"
format_stamp: "Formatted at 2026-08-10 07:21:17 on dist-test-slave-x5fp"
server_key: "6a1b90ef14592b77f0331c02c62ba31e"
server_key_iv: "5706421821b8f6e96d72096abd8b262b"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:17.126303 938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260810 07:21:17.144932 959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:17.145252 938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:21:17.145370 938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "d37a92c092954acfbf78271d06972e68"
format_stamp: "Formatted at 2026-08-10 07:21:17 on dist-test-slave-x5fp"
server_key: "6a1b90ef14592b77f0331c02c62ba31e"
server_key_iv: "5706421821b8f6e96d72096abd8b262b"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:17.145481 938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:17.163146 938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:17.167097 938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:17.167390 938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:17.167796 938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:21:17.168370 938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:21:17.168479 938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:17.168563 938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:21:17.168630 938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:17.180768 938 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:35533
I20260810 07:21:17.181350 938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:21:17.185050 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 938
I20260810 07:21:17.185158 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:21:17.185359 14992 external_mini_cluster.cc:1428] Setting key 4031bac53e73015dda193628ec018934
I20260810 07:21:17.188040 1140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:35533 every 8 connection(s)
Aug 10 07:21:17 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346477, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:21:17.201676 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.188933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:57323 (local address 127.14.164.62:40083)
0810 07:21:17.188996 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.188998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.189368 (+ 370us) server_negotiation.cc:407] Connection header received
0810 07:21:17.190357 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.190358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.190373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.190460 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:17.191274 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.192109 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.193810 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.194153 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.194241 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.197626 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:17.197628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:17.197628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:17.197637 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:17.199650 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:17.200348 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:17.200350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:17.200350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:17.200425 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:17.200802 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:17.200803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:17.200804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:17.200962 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:17.201038 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.201471 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.201572 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:21:17.201915 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.188950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:40083 (local address 127.14.164.3:57323)
0810 07:21:17.189194 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.189251 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.190190 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.190542 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:17.190547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:17.190819 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:21:17.191156 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.191163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.192234 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.192235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.193695 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.193698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.194262 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.194263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.194491 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.195323 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:21:17.195335 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:21:17.197504 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:21:17.199768 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:17.199779 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:17.199784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:17.200229 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:17.200512 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:17.200513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:17.200513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:17.200689 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:17.201044 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:21:17.201052 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:21:17.201275 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 07:21:17.201399 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:21:17.201612 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
I20260810 07:21:17.202972 1141 heartbeater.cc:347] Connected to a master server at 127.14.164.62:40083
I20260810 07:21:17.203110 1141 heartbeater.cc:464] Registering TS with master...
I20260810 07:21:17.203374 1141 heartbeater.cc:511] Master 127.14.164.62:40083 requested a full tablet report, sending...
I20260810 07:21:17.203848 386 ts_manager.cc:194] Registered new tserver with Master: d37a92c092954acfbf78271d06972e68 (127.14.164.3:35533)
I20260810 07:21:17.204351 386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:57323
I20260810 07:21:17.208962 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:21:17.219086 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.211008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.14.164.62:40083)
0810 07:21:17.211061 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.211063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.211272 (+ 209us) server_negotiation.cc:407] Connection header received
0810 07:21:17.211358 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.211358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.211371 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.211413 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:17.212171 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.212695 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.214018 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.214209 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.214276 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.216028 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:17.216029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:17.216030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:17.216037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:17.217800 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:17.218276 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:17.218277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:17.218277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:17.218336 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:17.218552 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:17.218552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:17.218552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:17.218659 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:17.218759 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.218938 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.219013 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:21:17.221874 386 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:21:17.223752 386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:21:17.233676 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.227950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.14.164.3:35533)
0810 07:21:17.228378 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.228382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.228394 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:21:17.228430 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.228432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.228446 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.228521 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:17.228708 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.231213 (+ 2505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.232263 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.233189 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.233286 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.233427 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.233505 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.233546 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":146,"threads_started":1}
I20260810 07:21:17.233641 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.227450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:35533 (local address 127.0.0.1:35988)
0810 07:21:17.227945 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.227991 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.228090 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.228526 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:17.228530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:17.228533 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:17.228612 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.228615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.231353 (+ 2738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.231354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.232144 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.232146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.233285 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.233286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.233366 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.233382 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:21:17.233489 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 07:21:17.233546 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260810 07:21:17.234543 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.227658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32941 (local address 127.0.0.1:35428)
0810 07:21:17.228200 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.228217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.228267 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.228774 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:17.228776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:17.228778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:17.228820 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.228821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.230225 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.230226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.232576 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.232579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.234285 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.234286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.234351 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.234356 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:21:17.234424 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:21:17.234463 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":1,"thread_start_us":192,"threads_started":1}
I20260810 07:21:17.234918 1037 tablet_service.cc:1467] Processing CreateTablet for tablet 8c094195b14748a3b9f68b4a425527be (DEFAULT_TABLE table=test-table [id=3a52f4515a67472bb08900746fdac9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:21:17.235107 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.228105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.14.164.1:32941)
0810 07:21:17.228285 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.228291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.228304 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:21:17.228345 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.228347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.228365 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.228445 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:17.228911 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.230128 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.232668 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.234112 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.234191 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.234250 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.234955 (+ 705us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.235003 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260810 07:21:17.235152 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.227450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:40001 (local address 127.0.0.1:53790)
0810 07:21:17.228656 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.228666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.228738 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.229113 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:17.229114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:17.229116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:17.229149 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.229151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.230406 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.230407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.231613 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.231616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.234305 (+ 2689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.234317 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.234438 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.234443 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:21:17.234930 (+ 487us) client_negotiation.cc:240] Negotiation successful
0810 07:21:17.235005 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"mutex_wait_us":334,"thread_start_us":115,"threads_started":1}
I20260810 07:21:17.235385 1037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c094195b14748a3b9f68b4a425527be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:17.235610 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.227591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.14.164.2:40001)
0810 07:21:17.228589 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.228595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.228802 (+ 207us) server_negotiation.cc:407] Connection header received
0810 07:21:17.228882 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.228885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.228903 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.228977 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:17.229239 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.230319 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.231830 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.234002 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.234114 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.234235 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.235374 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.235477 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":106,"threads_started":1}
I20260810 07:21:17.237567 618 tablet_service.cc:1467] Processing CreateTablet for tablet 8c094195b14748a3b9f68b4a425527be (DEFAULT_TABLE table=test-table [id=3a52f4515a67472bb08900746fdac9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:21:17.238123 618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c094195b14748a3b9f68b4a425527be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:17.238938 1167 tablet_bootstrap.cc:492] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Bootstrap starting.
I20260810 07:21:17.240036 1167 tablet_bootstrap.cc:654] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:17.240499 1167 log.cc:824] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:17.241644 834 tablet_service.cc:1467] Processing CreateTablet for tablet 8c094195b14748a3b9f68b4a425527be (DEFAULT_TABLE table=test-table [id=3a52f4515a67472bb08900746fdac9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:21:17.242015 834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c094195b14748a3b9f68b4a425527be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:17.244232 1169 tablet_bootstrap.cc:492] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: Bootstrap starting.
I20260810 07:21:17.245643 1170 tablet_bootstrap.cc:492] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Bootstrap starting.
I20260810 07:21:17.247181 1170 tablet_bootstrap.cc:654] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:17.247951 1170 log.cc:824] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:17.248980 1170 tablet_bootstrap.cc:492] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: No bootstrap required, opened a new log
I20260810 07:21:17.249051 1170 ts_tablet_manager.cc:1397] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:21:17.252386 1170 raft_consensus.cc:348] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.252836 1170 raft_consensus.cc:374] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:17.252928 1170 raft_consensus.cc:729] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578267532eb747c499ece61881bdedf6, State: Initialized, Role: FOLLOWER
I20260810 07:21:17.253113 1170 consensus_queue.cc:260] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.253648 1170 ts_tablet_manager.cc:1428] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:21:17.253917 929 heartbeater.cc:503] Master 127.14.164.62:40083 was elected leader, sending a full tablet report...
I20260810 07:21:17.258046 1169 tablet_bootstrap.cc:654] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:17.259140 1169 log.cc:824] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:17.259871 1167 tablet_bootstrap.cc:492] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: No bootstrap required, opened a new log
I20260810 07:21:17.259964 1167 ts_tablet_manager.cc:1397] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20260810 07:21:17.262959 1167 raft_consensus.cc:348] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.263149 1167 raft_consensus.cc:374] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:17.263185 1167 raft_consensus.cc:729] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37a92c092954acfbf78271d06972e68, State: Initialized, Role: FOLLOWER
I20260810 07:21:17.263446 1167 consensus_queue.cc:260] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.263868 1167 ts_tablet_manager.cc:1428] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260810 07:21:17.264045 1141 heartbeater.cc:503] Master 127.14.164.62:40083 was elected leader, sending a full tablet report...
I20260810 07:21:17.266218 1169 tablet_bootstrap.cc:492] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: No bootstrap required, opened a new log
I20260810 07:21:17.266379 1169 ts_tablet_manager.cc:1397] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260810 07:21:17.270578 1169 raft_consensus.cc:348] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.270895 1169 raft_consensus.cc:374] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:17.271004 1169 raft_consensus.cc:729] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fe940bc2754e4a80f0f3630fb6e3a2, State: Initialized, Role: FOLLOWER
I20260810 07:21:17.271214 1169 consensus_queue.cc:260] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.271764 715 heartbeater.cc:503] Master 127.14.164.62:40083 was elected leader, sending a full tablet report...
I20260810 07:21:17.272068 1169 ts_tablet_manager.cc:1428] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260810 07:21:17.332785 717 tablet.cc:2369] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:21:17.372913 930 tablet.cc:2369] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 07:21:17 dist-test-slave-x5fp krb5kdc[32144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346476, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
W20260810 07:21:17.439059 1142 tablet.cc:2369] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:21:17.439952 1176 raft_consensus.cc:482] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:21:17.440075 1176 raft_consensus.cc:504] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.440660 1176 leader_election.cc:288] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 578267532eb747c499ece61881bdedf6 (127.14.164.2:40001), d9fe940bc2754e4a80f0f3630fb6e3a2 (127.14.164.1:32941)
I20260810 07:21:17.445616 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.440946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:40001 (local address 127.14.164.3:52803)
0810 07:21:17.441087 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.441099 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.441144 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.442122 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:17.442124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:17.442126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:17.442340 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.442343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.443243 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.443245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.444386 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.444390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.445279 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.445280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.445433 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.445437 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:21:17.445493 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:21:17.445538 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 07:21:17.446415 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.441306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:52803 (local address 127.14.164.2:40001)
0810 07:21:17.441387 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.441390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.441398 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:21:17.441432 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.441433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.441442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.441490 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:17.442480 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.443147 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.444483 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.445136 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.445232 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.445243 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.446303 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.446348 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:21:17.446729 859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c094195b14748a3b9f68b4a425527be" candidate_uuid: "d37a92c092954acfbf78271d06972e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578267532eb747c499ece61881bdedf6" is_pre_election: true
I20260810 07:21:17.446923 859 raft_consensus.cc:2454] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d37a92c092954acfbf78271d06972e68 in term 0.
I20260810 07:21:17.447896 963 leader_election.cc:302] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578267532eb747c499ece61881bdedf6, d37a92c092954acfbf78271d06972e68; no voters:
I20260810 07:21:17.448173 1176 raft_consensus.cc:2781] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:21:17.448257 1176 raft_consensus.cc:482] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:21:17.448288 1176 raft_consensus.cc:3037] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:17.449569 1176 raft_consensus.cc:504] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.449736 1176 leader_election.cc:288] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [CANDIDATE]: Term 1 election: Requested vote from peers 578267532eb747c499ece61881bdedf6 (127.14.164.2:40001), d9fe940bc2754e4a80f0f3630fb6e3a2 (127.14.164.1:32941)
I20260810 07:21:17.450119 859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c094195b14748a3b9f68b4a425527be" candidate_uuid: "d37a92c092954acfbf78271d06972e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578267532eb747c499ece61881bdedf6"
I20260810 07:21:17.450234 859 raft_consensus.cc:3037] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:17.451167 859 raft_consensus.cc:2454] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d37a92c092954acfbf78271d06972e68 in term 1.
I20260810 07:21:17.451719 963 leader_election.cc:302] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578267532eb747c499ece61881bdedf6, d37a92c092954acfbf78271d06972e68; no voters:
I20260810 07:21:17.452860 1176 raft_consensus.cc:2781] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:21:17.453092 1176 raft_consensus.cc:686] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 1 LEADER]: Becoming Leader. State: Replica: d37a92c092954acfbf78271d06972e68, State: Running, Role: LEADER
I20260810 07:21:17.453218 1176 consensus_queue.cc:237] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } }
I20260810 07:21:17.454514 385 catalog_manager.cc:5495] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 reported cstate change: term changed from 0 to 1, leader changed from <none> to d37a92c092954acfbf78271d06972e68 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "d37a92c092954acfbf78271d06972e68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d37a92c092954acfbf78271d06972e68" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 } health_report { overall_health: UNKNOWN } } }
I20260810 07:21:17.455382 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.441809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32941 (local address 127.14.164.3:39329)
0810 07:21:17.445708 (+ 3899us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.445720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.445765 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.446782 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:17.446783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:17.446785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:17.446823 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.446824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.450047 (+ 3223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.450048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.451533 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:17.451536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.455101 (+ 3565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.455103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:17.455175 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.455180 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:21:17.455264 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:21:17.455312 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3886,"thread_start_us":117,"threads_started":1}
I20260810 07:21:17.455735 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.442038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:39329 (local address 127.14.164.1:32941)
0810 07:21:17.442089 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.442092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.445824 (+ 3732us) server_negotiation.cc:407] Connection header received
0810 07:21:17.445859 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.445860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.445876 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.445925 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:17.446918 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.447625 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.451867 (+ 4242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.452672 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.452812 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.452825 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.455595 (+ 2770us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.455652 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:21:17.456347 651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c094195b14748a3b9f68b4a425527be" candidate_uuid: "d37a92c092954acfbf78271d06972e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" is_pre_election: true
I20260810 07:21:17.456523 651 raft_consensus.cc:2454] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d37a92c092954acfbf78271d06972e68 in term 0.
I20260810 07:21:17.456660 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c094195b14748a3b9f68b4a425527be" candidate_uuid: "d37a92c092954acfbf78271d06972e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2"
I20260810 07:21:17.456784 649 raft_consensus.cc:3037] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:17.458127 649 raft_consensus.cc:2454] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d37a92c092954acfbf78271d06972e68 in term 1.
I20260810 07:21:17.469025 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.465197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.14.164.3:35533)
0810 07:21:17.465241 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:17.465243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:17.465253 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:21:17.465335 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:17.465336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:17.465361 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:17.465435 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:21:17.465687 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.466480 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.467666 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:17.468065 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:17.468134 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:17.468363 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:21:17.468528 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:21:17.468616 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:17.468831 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:21:17.468853 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:21:17.471478 649 raft_consensus.cc:1261] T 8c094195b14748a3b9f68b4a425527be P d9fe940bc2754e4a80f0f3630fb6e3a2 [term 1 FOLLOWER]: Refusing update from remote peer d37a92c092954acfbf78271d06972e68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:21:17.471984 1182 consensus_queue.cc:1035] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9fe940bc2754e4a80f0f3630fb6e3a2" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 32941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:21:17.474682 859 raft_consensus.cc:1261] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 1 FOLLOWER]: Refusing update from remote peer d37a92c092954acfbf78271d06972e68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:21:17.475018 1182 consensus_queue.cc:1035] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578267532eb747c499ece61881bdedf6" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:21:17.479857 1185 mvcc.cc:205] Tried to move back new op lower bound from 7316875171719888896 to 7316875171649122304. Current Snapshot: MvccSnapshot[applied={T|T < 7316875171719888896}]
I20260810 07:21:17.480149 1186 mvcc.cc:205] Tried to move back new op lower bound from 7316875171719888896 to 7316875171649122304. Current Snapshot: MvccSnapshot[applied={T|T < 7316875171719888896}]
I20260810 07:21:17.487869 385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:21:17.488016 385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:21:17.489156 385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0cb071ade0b4baf9c28efc083b6ddff: Sending DeleteTablet for 3 replicas of tablet 8c094195b14748a3b9f68b4a425527be
I20260810 07:21:17.489810 834 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c094195b14748a3b9f68b4a425527be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:21:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:53790
I20260810 07:21:17.490253 618 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c094195b14748a3b9f68b4a425527be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:21:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:35428
I20260810 07:21:17.492987 1037 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c094195b14748a3b9f68b4a425527be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:21:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:35988
I20260810 07:21:17.493860 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 522
I20260810 07:21:17.508884 1201 tablet_replica.cc:330] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: stopping tablet replica
I20260810 07:21:17.509063 1201 raft_consensus.cc:2229] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:21:17.509224 1201 raft_consensus.cc:2258] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:21:17.509624 1196 tablet_replica.cc:330] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: stopping tablet replica
I20260810 07:21:17.509785 1196 raft_consensus.cc:2229] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:21:17.509871 1196 pending_rounds.cc:62] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Trying to abort 2 pending ops.
I20260810 07:21:17.509931 1196 pending_rounds.cc:68] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316875171719888896 op_type: WRITE_OP write_request { tablet_id: "8c094195b14748a3b9f68b4a425527be" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\231\372\345\323\006\"8\n\ntest-admin\022*\n 3a52f4515a67472bb08900746fdac9f2\020\001\030\001 \001(\001" signature: "<redacted>""\2360\251\210\265\3659\035-\371\272\255\372g\271\261y\267af\030\016\320\335\371\343\267\\\325\225\223r\316\356\327\317\020\362QM\t#\257\307\255Pn\346p\232\277P\034\233\000d\272\214\206i\211I\250I" signing_key_seq_num: 0 } } request_id { client_id: "20f4271f4bfd4a859bf202ba928fa08e" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260810 07:21:17.510318 1196 pending_rounds.cc:68] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316875171649122304 op_type: NO_OP noop_request { }
I20260810 07:21:17.510393 1196 raft_consensus.cc:2866] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:21:17.510428 1196 raft_consensus.cc:2258] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:21:17.511039 1201 ts_tablet_manager.cc:1905] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:21:17.513259 1201 ts_tablet_manager.cc:1918] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:21:17.513407 1201 log.cc:1196] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/8c094195b14748a3b9f68b4a425527be
I20260810 07:21:17.513779 1201 ts_tablet_manager.cc:1939] T 8c094195b14748a3b9f68b4a425527be P d37a92c092954acfbf78271d06972e68: Deleting consensus metadata
I20260810 07:21:17.514323 360 catalog_manager.cc:4841] TS d37a92c092954acfbf78271d06972e68 (127.14.164.3:35533): tablet 8c094195b14748a3b9f68b4a425527be (table test-table [id=3a52f4515a67472bb08900746fdac9f2]) successfully deleted
W20260810 07:21:17.521268 364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:32941 (error 108)
I20260810 07:21:17.522099 1196 ts_tablet_manager.cc:1905] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:21:17.524461 1196 ts_tablet_manager.cc:1918] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:21:17.525048 1196 log.cc:1196] T 8c094195b14748a3b9f68b4a425527be P 578267532eb747c499ece61881bdedf6: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/8c094195b14748a3b9f68b4a425527be
I20260810 07:21:17.525287 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 727
I20260810 07:21:17.581837 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.576540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32941 (local address 127.0.0.1:35438)
0810 07:21:17.581675 (+ 5135us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.581775 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:32941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5093}
W20260810 07:21:17.583247 364 catalog_manager.cc:4584] TS d9fe940bc2754e4a80f0f3630fb6e3a2 (127.14.164.1:32941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c094195b14748a3b9f68b4a425527be: Network error: Client connection negotiation failed: client connection to 127.14.164.1:32941: connect: Connection refused (error 111)
W20260810 07:21:17.589290 1154 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:21:17.589036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:40001 (local address 127.0.0.1:53792)
0810 07:21:17.589086 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.589099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:17.589150 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:17.589212 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.2:40001: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 07:21:17.589533 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 938
I20260810 07:21:17.635979 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.635820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32941 (local address 127.0.0.1:35444)
0810 07:21:17.635886 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.635925 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:32941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:21:17.636523 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.636428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:32941 (local address 127.0.0.1:35450)
0810 07:21:17.636466 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.636491 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:32941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 07:21:17.651120 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.650973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:40001 (local address 127.0.0.1:53796)
0810 07:21:17.651035 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.651069 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.2:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:21:17.651638 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:17.651563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:40001 (local address 127.0.0.1:53798)
0810 07:21:17.651593 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:17.651614 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.2:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 07:21:17.658932 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 332
I20260810 07:21:17.684767 14992 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:21:18.040012 14992 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:21:18.040156 14992 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:21:18.395555 14992 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:21:18.395 UTC [32185] LOG: received smart shutdown request
2026-08-10 07:21:18.400 UTC [32185] LOG: background worker "logical replication launcher" (PID 32199) exited with exit code 1
2026-08-10 07:21:18.400 UTC [32190] LOG: shutting down
2026-08-10 07:21:18.608 UTC [32185] LOG: database system is shut down
2026-08-10T07:21:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53435 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:21:18 dist-test-slave-x5fp krb5kdc[1221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:21:18 dist-test-slave-x5fp krb5kdc[1221](info): set up 2 sockets
Aug 10 07:21:18 dist-test-slave-x5fp krb5kdc[1221](info): commencing operation
krb5kdc: starting...
W20260810 07:21:20.083933 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:21:20 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:21:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:21:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/rangeradmin_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/rangeradmin_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/rangerlookup_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/rangerlookup_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
I20260810 07:21:20.272505 14992 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMAuLWb/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/postgres -l logfile start
2026-08-10 07:21:23.005 UTC [1266] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:21:23.005 UTC [1266] LOG: listening on IPv4 address "127.14.164.20", port 47797
2026-08-10 07:21:23.009 UTC [1266] LOG: listening on Unix socket "/tmp/.s.PGSQL.47797"
2026-08-10 07:21:23.049 UTC [1274] LOG: database system was shut down at 2026-08-10 07:21:21 UTC
2026-08-10 07:21:23.053 UTC [1266] LOG: database system is ready to accept connections
I20260810 07:21:24.217559 14992 mini_postgres.cc:96] Postgres bound to 47797
2026-08-10 07:21:24.225 UTC [1287] FATAL: database "slave" does not exist
127.14.164.20:47797 - accepting connections
I20260810 07:21:24.226028 14992 mini_ranger.cc:161] Starting Ranger...
I20260810 07:21:24.242566 14992 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:21:24.826957 14992 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:21:24.827057 14992 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-admin
2026-08-10 07:21:25,129 [I] DB FLAVOR :POSTGRES
2026-08-10 07:21:25,129 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:21:25,129 [I] Checking connection..
2026-08-10 07:21:25,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:21:25,411 [I] Checking connection passed.
2026-08-10 07:21:25,412 [I] --------- Verifying version history table ---------
2026-08-10 07:21:25,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:25,695 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:21:25,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:25,985 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:21:25,985 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:21:25,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:21:26.244 UTC [1379] WARNING: there is no transaction in progress
2026-08-10 07:21:26,255 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:21:26,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:26,514 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:21:26,515 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:21:26,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:21:26,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:21:27,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x5fp.c.gcp-upstream.internal','N') ;"
2026-08-10 07:21:27,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:21:27,555 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:21:27,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:21:27,806 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:21:27,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:21:28,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:21:28,350 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:21:28,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:21:29.364 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.382 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.399 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.415 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.433 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.453 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.513 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.521 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.537 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.549 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.561 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.577 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.588 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.595 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.602 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:29.610 UTC [1543] WARNING: there is no transaction in progress
2026-08-10 07:21:30,304 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:21:30,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:21:30,571 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:21:30,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:21:30,837 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:21:30,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:21:31,105 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:21:31,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:21:31,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:21:31,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x5fp.c.gcp-upstream.internal';"
2026-08-10 07:21:31,902 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:21:31,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:21:32,173 [I] DB_PATCHES have already been applied
I20260810 07:21:32.179841 14992 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMAuLWb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:21:32.179916 14992 mini_ranger.cc:193] Using host: 127.14.164.20
I20260810 07:21:32.184140 14992 mini_ranger.cc:227] Ranger admin URL: http://127.14.164.20:49697
Aug 10, 2026 7:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:21:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:21:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:21:33.561839 14992 mini_ranger.cc:160] Time spent starting Ranger: real 9.336s user 0.000s sys 0.011s
Aug 10, 2026 7:21:54 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:21:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 7:21:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:21:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:21:55 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:21:56.376852 14992 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.164.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.164.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/rangerkms_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.164.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/rangerkms_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.164.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.164.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.164.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/HTTP_127.14.164.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:21:56 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346516, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:21:56.490635 14992 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:21:56.503800 14992 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:21:57.510339 14992 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:21:57.510423 14992 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms
I20260810 07:21:57.510439 14992 mini_ranger_kms.cc:227] Using postgres at 127.14.164.20:47797
2026-08-10 07:21:57,709 [I] DB FLAVOR :POSTGRES
2026-08-10 07:21:57,709 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:21:57,709 [I] Checking connection
2026-08-10 07:21:57,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:21:57,975 [I] connection success
2026-08-10 07:21:57,975 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:21:57,975 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:21:57,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:21:58,230 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:21:58,230 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:21:58,231 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:21:58,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.164.20:47797/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:21:58,514 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:21:59.046391 14992 mini_ranger_kms.cc:310] Created kms service
I20260810 07:21:59.241387 14992 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:21:59.316586 14992 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:21:59 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346516, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:21:59.786296 14992 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:21:59.786429 14992 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMAuLWb/build/release/bin/postgresql.jar:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMAuLWb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:21:59.786450 14992 mini_ranger_kms.cc:240] Using host: 127.14.164.20
I20260810 07:21:59.791500 14992 mini_ranger_kms.cc:276] Ranger KMS PID: 1827
I20260810 07:21:59.791595 14992 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.164.20:41003
Aug 10, 2026 7:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:22:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:22:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:22:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:22:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:22:02 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346522, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.164.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:02.779690 14992 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.289s user 0.004s sys 0.179s
I20260810 07:22:02.779827 14992 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:22:02.779881 14992 mini_ranger_kms.cc:401] 127.14.164.20:41003/kms/v1/keys
Aug 10 07:22:02 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346522, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.164.20@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
Aug 10 07:22:03 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346522, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.164.20@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
Aug 10 07:22:05 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:22:05.721933 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35525
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35525
--ranger_config_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:05.825984 1908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:05.826216 1908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:05.826257 1908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:05.827472 1908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:05.827521 1908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:05.827535 1908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:05.827548 1908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:05.827560 1908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:05.829360 1908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.164.20:41003/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35525
--ranger_config_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:05.829629 1908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:05.829896 1908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:05.835005 1916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:05.834968 1917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:05.834978 1920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:05.835335 1908 server_base.cc:1034] running on GCE node
I20260810 07:22:05.835649 1908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:05.836004 1908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:05.837167 1908 hybrid_clock.cc:648] HybridClock initialized: now 1786346525837156 us; error 43 us; skew 500 ppm
Aug 10 07:22:05 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346525, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:05.840075 1908 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:05.840536 1908 webserver.cc:460] Webserver started at http://127.14.164.62:41173/ using document root <none> and password file <none>
I20260810 07:22:05.840847 1908 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:05.840910 1908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:05.841077 1908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:05 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346525, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:05.907104 1908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "1c4b70b00cbb4978a35b40731f10eb22"
format_stamp: "Formatted at 2026-08-10 07:22:05 on dist-test-slave-x5fp"
server_key: "4074233146f37e691342b09c871c6f6c"
server_key_iv: "df96c740136a5a6d52646fba8663f912"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:05.907647 1908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "1c4b70b00cbb4978a35b40731f10eb22"
format_stamp: "Formatted at 2026-08-10 07:22:05 on dist-test-slave-x5fp"
server_key: "4074233146f37e691342b09c871c6f6c"
server_key_iv: "df96c740136a5a6d52646fba8663f912"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:05.909776 1908 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260810 07:22:05.925766 1932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:05.926115 1908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:05.926250 1908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "1c4b70b00cbb4978a35b40731f10eb22"
format_stamp: "Formatted at 2026-08-10 07:22:05 on dist-test-slave-x5fp"
server_key: "4074233146f37e691342b09c871c6f6c"
server_key_iv: "df96c740136a5a6d52646fba8663f912"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:05.926362 1908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:05.947348 1908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:05.950017 1908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:05.950201 1908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:05.955924 1908 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:35525
I20260810 07:22:05.956368 2018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:35525 every 8 connection(s)
I20260810 07:22:05.956465 1908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:05.957763 2019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:05.961241 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 1908
I20260810 07:22:05.961364 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:22:05.961565 14992 external_mini_cluster.cc:1428] Setting key 6a5e091b6cd9544339689ab6ad364546
I20260810 07:22:05.962335 2019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Bootstrap starting.
I20260810 07:22:05.963075 2019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:05.963416 2019 log.cc:824] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:05.964150 2019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: No bootstrap required, opened a new log
I20260810 07:22:05.966074 2019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:05.966260 2019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:05.966293 2019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4b70b00cbb4978a35b40731f10eb22, State: Initialized, Role: FOLLOWER
I20260810 07:22:05.966415 2019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:05.966495 2019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:05.966528 2019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:05.966566 2019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:05.967267 2019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:05.967393 2019 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4b70b00cbb4978a35b40731f10eb22; no voters:
I20260810 07:22:05.967617 2019 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:05.967650 2031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:05.967780 2031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 LEADER]: Becoming Leader. State: Replica: 1c4b70b00cbb4978a35b40731f10eb22, State: Running, Role: LEADER
I20260810 07:22:05.967866 2031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:05.967940 2019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:05.968186 2031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c4b70b00cbb4978a35b40731f10eb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } } }
I20260810 07:22:05.968258 2031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:05.968384 2031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4b70b00cbb4978a35b40731f10eb22. Latest consensus state: current_term: 1 leader_uuid: "1c4b70b00cbb4978a35b40731f10eb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } } }
I20260810 07:22:05.968468 2031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:05.968997 2037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:05.969624 2037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:05.969951 2019 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 10 07:22:05 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:05.979338 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:05.962399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.14.164.62:35525)
0810 07:22:05.963226 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:05.963238 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:05.963287 (+ 49us) server_negotiation.cc:407] Connection header received
0810 07:22:05.963919 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:05.963932 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:05.964005 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:05.964816 (+ 811us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:05.965111 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:05.966913 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:05.968739 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:05.969443 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:05.969569 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:05.973461 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:05.973514 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:05.973515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:05.973633 (+ 118us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:05.976787 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:05.977247 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:05.977251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:05.977251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:05.977357 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:05.977575 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:05.977576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:05.977576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:05.977947 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:05.978105 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:05.978680 (+ 575us) server_negotiation.cc:299] Negotiation successful
0810 07:22:05.978866 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":91,"threads_started":1}
I20260810 07:22:05.986070 2055 mvcc.cc:205] Tried to move back new op lower bound from 7316875370376409088 to 7316875370364735488. Current Snapshot: MvccSnapshot[applied={T|T < 7316875370376409088}]
I20260810 07:22:05.989627 2037 catalog_manager.cc:1347] Generated new cluster ID: 456c4ed2ca2a414ca724169ce5688d15
I20260810 07:22:05.989692 2037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:06.016280 2037 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:06.017242 2037 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:06.034368 2037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Generated new TSK 0
I20260810 07:22:06.034708 2037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:22:06.749003 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35525
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:06.898343 2095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:06.898622 2095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:06.898665 2095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:06.899981 2095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:06.900074 2095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:06.900123 2095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:06.902014 2095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.164.20:41003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:06.902338 2095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:06.902607 2095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:06.910207 2109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:06.910298 2104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:06.910638 2106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:06.911424 2095 server_base.cc:1034] running on GCE node
I20260810 07:22:06.911729 2095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:06.912083 2095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:06.913376 2095 hybrid_clock.cc:648] HybridClock initialized: now 1786346526913347 us; error 40 us; skew 500 ppm
Aug 10 07:22:06 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346526, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:06.917964 2095 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:06.918526 2095 webserver.cc:460] Webserver started at http://127.14.164.1:37775/ using document root <none> and password file <none>
I20260810 07:22:06.918828 2095 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:06.918907 2095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:06.919065 2095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:06 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346526, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:06.937086 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee"
format_stamp: "Formatted at 2026-08-10 07:22:06 on dist-test-slave-x5fp"
server_key: "2b5d5fa9d4b5d2fe2426684d36ab6954"
server_key_iv: "1108fecc1a887a6bb91122c08ceb90a3"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:06.937973 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee"
format_stamp: "Formatted at 2026-08-10 07:22:06 on dist-test-slave-x5fp"
server_key: "2b5d5fa9d4b5d2fe2426684d36ab6954"
server_key_iv: "1108fecc1a887a6bb91122c08ceb90a3"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:06.940625 2095 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:06.954550 2121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:06.954845 2095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:22:06.954972 2095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee"
format_stamp: "Formatted at 2026-08-10 07:22:06 on dist-test-slave-x5fp"
server_key: "2b5d5fa9d4b5d2fe2426684d36ab6954"
server_key_iv: "1108fecc1a887a6bb91122c08ceb90a3"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:06.955104 2095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:06.969637 2095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:06.972599 2095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:06.972813 2095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:06.973137 2095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:06.973579 2095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:06.973645 2095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:06.973727 2095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:06.973773 2095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:06.988808 2095 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:33343
I20260810 07:22:06.989503 2095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:06.999282 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 2095
I20260810 07:22:06.999392 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:22:06.999596 14992 external_mini_cluster.cc:1428] Setting key 01777583fe9ff8d40e0c42671c81437e
I20260810 07:22:06.996086 2313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:33343 every 8 connection(s)
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346526, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
I20260810 07:22:07.013016 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:06.997476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:38823 (local address 127.14.164.62:35525)
0810 07:22:06.997810 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:06.997813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:06.998035 (+ 222us) server_negotiation.cc:407] Connection header received
0810 07:22:06.999001 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:06.999002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:06.999017 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:07.000947 (+ 1930us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:07.002242 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.003117 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.005000 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.005343 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.005409 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:07.008773 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:07.008775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:07.008776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:07.008796 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:07.010823 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:07.011574 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:07.011576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:07.011576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:07.011662 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:07.012021 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:07.012023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:07.012023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:07.012205 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:07.012259 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:07.012834 (+ 575us) server_negotiation.cc:299] Negotiation successful
0810 07:22:07.012931 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":91,"threads_started":1}
I20260810 07:22:07.013299 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:06.997014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35525 (local address 127.14.164.1:38823)
0810 07:22:06.997893 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:06.997944 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:06.998862 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:07.001242 (+ 2380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:07.001248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:07.001692 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:07.002089 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:07.002096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.003285 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.003286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:07.004886 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:07.004891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.005583 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.005584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:07.005874 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:07.006618 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:07.006630 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:07.008598 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:07.011097 (+ 2499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:07.011107 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:07.011110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:07.011440 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:07.011746 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:07.011746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:07.011747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:07.011930 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:07.012316 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:07.012323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:07.012548 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 07:22:07.012683 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 07:22:07.012981 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":202,"threads_started":1}
I20260810 07:22:07.014169 2314 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35525
I20260810 07:22:07.014313 2314 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:07.014652 2314 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
I20260810 07:22:07.015445 1962 ts_manager.cc:194] Registered new tserver with Master: 126b8ed7f1dd4abbb23c974fb0ec2bee (127.14.164.1:33343)
I20260810 07:22:07.016180 1962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:38823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:22:07.036511 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35525
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:07.182173 2324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:07.182574 2324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:07.182642 2324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:07.184552 2324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:07.184639 2324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:07.184691 2324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:07.187134 2324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.164.20:41003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:07.187463 2324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:07.187865 2324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:07.195942 2334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.196115 2337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.195940 2339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:07.197690 2324 server_base.cc:1034] running on GCE node
I20260810 07:22:07.197988 2324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:07.198341 2324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:07.199573 2324 hybrid_clock.cc:648] HybridClock initialized: now 1786346527199548 us; error 39 us; skew 500 ppm
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:07.203752 2324 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:07.204375 2324 webserver.cc:460] Webserver started at http://127.14.164.2:42003/ using document root <none> and password file <none>
I20260810 07:22:07.204659 2324 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:07.204789 2324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:07.204972 2324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:07.220639 2324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "615d086687ae4a248c91dacbf348baed"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c761092af420333186e8fe08786d8510"
server_key_iv: "0de5687873961254f4bc835937b02118"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.222226 2324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "615d086687ae4a248c91dacbf348baed"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c761092af420333186e8fe08786d8510"
server_key_iv: "0de5687873961254f4bc835937b02118"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.224812 2324 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260810 07:22:07.241497 2347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.241809 2324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:07.241945 2324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "615d086687ae4a248c91dacbf348baed"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c761092af420333186e8fe08786d8510"
server_key_iv: "0de5687873961254f4bc835937b02118"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.242072 2324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:07.265039 2324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:07.267922 2324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:07.268117 2324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:07.268359 2324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:07.268833 2324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:07.268900 2324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.268934 2324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:07.268997 2324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.280426 2324 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:44131
I20260810 07:22:07.280894 2518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:44131 every 8 connection(s)
I20260810 07:22:07.280951 2324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:07.284807 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 2324
I20260810 07:22:07.284941 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:22:07.285141 14992 external_mini_cluster.cc:1428] Setting key ed4b2300de0a191bacc2d4225247af3a
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:07.294196 2522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:07.281927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35525 (local address 127.14.164.2:33129)
0810 07:22:07.282208 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:07.282253 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:07.282883 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:07.283154 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:07.283159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:07.283439 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:07.283689 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:07.283694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.285410 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.285412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:07.286766 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:07.286772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.287315 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.287316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:07.287425 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:07.288160 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:07.288170 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:07.290064 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:07.292244 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:07.292253 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:07.292256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:07.292542 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:07.292801 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:07.292802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:07.292802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:07.292937 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:07.293353 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:07.293361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:07.293556 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 07:22:07.293678 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:22:07.293864 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":193,"threads_started":1}
I20260810 07:22:07.294586 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:07.281945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:33129 (local address 127.14.164.62:35525)
0810 07:22:07.282044 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:07.282047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:07.282396 (+ 349us) server_negotiation.cc:407] Connection header received
0810 07:22:07.283000 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:07.283001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:07.283018 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:07.283080 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:07.283855 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.284448 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.286921 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.287177 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.287221 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:07.290180 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:07.290182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:07.290182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:07.290191 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:07.292134 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:07.292631 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:07.292632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:07.292633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:07.292696 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:07.293023 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:07.293024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:07.293024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:07.293242 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:07.293321 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:07.294377 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0810 07:22:07.294482 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:22:07.295076 2519 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35525
I20260810 07:22:07.295209 2519 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:07.295450 2519 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:07.295918 1962 ts_manager.cc:194] Registered new tserver with Master: 615d086687ae4a248c91dacbf348baed (127.14.164.2:44131)
I20260810 07:22:07.296450 1962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:33129
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:22:07.324558 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35525
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:07.447682 2528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:07.447995 2528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:07.448038 2528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:07.449970 2528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:07.450038 2528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:07.450083 2528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:07.452467 2528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.164.20:41003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:07.452751 2528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:07.453161 2528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:07.461385 2528 server_base.cc:1034] running on GCE node
W20260810 07:22:07.461615 2533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.462958 2539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.463188 2536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:07.463563 2528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:07.463922 2528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:07.465101 2528 hybrid_clock.cc:648] HybridClock initialized: now 1786346527465075 us; error 41 us; skew 500 ppm
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:07.468839 2528 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:07.469465 2528 webserver.cc:460] Webserver started at http://127.14.164.3:42849/ using document root <none> and password file <none>
I20260810 07:22:07.469836 2528 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:07.469919 2528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:07.470080 2528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:07.491045 2528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "c2bd92dba909418e86ba2d4a6d75a3d0"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c6767d91be797082702485fdcd0afeee"
server_key_iv: "64762d7cbde485df5f24e51e6b7ce70b"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.491554 2528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "c2bd92dba909418e86ba2d4a6d75a3d0"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c6767d91be797082702485fdcd0afeee"
server_key_iv: "64762d7cbde485df5f24e51e6b7ce70b"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.494608 2528 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:07.513334 2552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.513607 2528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:07.513768 2528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "c2bd92dba909418e86ba2d4a6d75a3d0"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c6767d91be797082702485fdcd0afeee"
server_key_iv: "64762d7cbde485df5f24e51e6b7ce70b"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.513902 2528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:07.528554 2528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:07.531147 2528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:07.531342 2528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:07.531688 2528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:07.532222 2528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:07.532266 2528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.532292 2528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:07.532341 2528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.546141 2528 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:40931
I20260810 07:22:07.546360 2716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:40931 every 8 connection(s)
I20260810 07:22:07.546694 2528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:07.554600 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 2528
I20260810 07:22:07.554850 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:22:07.555190 14992 external_mini_cluster.cc:1428] Setting key ec5c57bb94535aa85a0eafd7e720d4c4
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:07.559921 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:07.548124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:41679 (local address 127.14.164.62:35525)
0810 07:22:07.548207 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:07.548209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:07.548223 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:22:07.548935 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:07.548937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:07.548961 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:07.549055 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:07.550243 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.550967 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.552358 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.552876 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.553013 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:07.556491 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:07.556493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:07.556493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:07.556502 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:07.558278 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:07.558788 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:07.558790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:07.558790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:07.558858 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:07.559181 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:07.559182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:07.559182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:07.559315 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:07.559397 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:07.559727 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 07:22:07.559821 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:22:07.560241 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:07.547741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35525 (local address 127.14.164.3:41679)
0810 07:22:07.548015 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:07.548090 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:07.548804 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:07.549294 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:07.549301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:07.549658 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:07.550075 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:07.550082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.551124 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.551126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:07.552235 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:07.552238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:07.553040 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:07.553041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:07.553219 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:07.554301 (+ 1082us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:07.554314 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:07.556368 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:07.558407 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:07.558416 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:07.558419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:07.558684 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:07.558948 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:07.558949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:07.558949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:07.559117 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:07.559389 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:07.559397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:07.559562 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 07:22:07.559683 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 07:22:07.559882 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
I20260810 07:22:07.560963 2717 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35525
I20260810 07:22:07.561110 2717 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:07.561321 2717 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:07.561825 1962 ts_manager.cc:194] Registered new tserver with Master: c2bd92dba909418e86ba2d4a6d75a3d0 (127.14.164.3:40931)
I20260810 07:22:07.562250 1962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:41679
I20260810 07:22:07.568087 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:07.568974 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 2095
I20260810 07:22:07.586625 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 2324
I20260810 07:22:07.604502 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 2528
I20260810 07:22:07.625329 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 1908
I20260810 07:22:07.639429 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35525
--webserver_interface=127.14.164.62
--webserver_port=41173
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35525
--ranger_config_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:07.724458 2721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:07.724762 2721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:07.724815 2721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:07.726003 2721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:07.726078 2721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:07.726104 2721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:07.726136 2721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:07.726188 2721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:07.727890 2721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35525
--ranger_config_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=41173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:07.728188 2721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:07.728425 2721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:07.733503 2727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.733521 2728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:07.733685 2721 server_base.cc:1034] running on GCE node
W20260810 07:22:07.733691 2731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:07.734112 2721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:07.734434 2721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:07.735607 2721 hybrid_clock.cc:648] HybridClock initialized: now 1786346527735597 us; error 44 us; skew 500 ppm
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:07.738215 2721 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:07.738770 2721 webserver.cc:460] Webserver started at http://127.14.164.62:41173/ using document root <none> and password file <none>
I20260810 07:22:07.739013 2721 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:07.739095 2721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:07.785300 2721 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:07.786219 2741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.786445 2721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:07.786563 2721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "1c4b70b00cbb4978a35b40731f10eb22"
format_stamp: "Formatted at 2026-08-10 07:22:05 on dist-test-slave-x5fp"
server_key: "4074233146f37e691342b09c871c6f6c"
server_key_iv: "df96c740136a5a6d52646fba8663f912"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.787086 2721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:07.800164 2721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:07.802098 2721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:07.802284 2721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:07.806840 2721 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:35525
I20260810 07:22:07.806887 2809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:35525 every 8 connection(s)
I20260810 07:22:07.807264 2721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:07.808957 2810 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:07.809625 2810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Bootstrap starting.
I20260810 07:22:07.816663 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 2721
I20260810 07:22:07.817066 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:33343
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:35525
--webserver_port=37775
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:07.818383 2810 log.cc:824] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:07.820228 2810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:07.820538 2810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Bootstrap complete.
I20260810 07:22:07.823153 2810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:07.823419 2810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4b70b00cbb4978a35b40731f10eb22, State: Initialized, Role: FOLLOWER
I20260810 07:22:07.823638 2810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:07.823740 2810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:07.823796 2810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:07.823845 2810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:07.824937 2810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:07.825065 2810 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4b70b00cbb4978a35b40731f10eb22; no voters:
I20260810 07:22:07.825275 2810 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:22:07.825314 2816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:22:07.825438 2816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [term 2 LEADER]: Becoming Leader. State: Replica: 1c4b70b00cbb4978a35b40731f10eb22, State: Running, Role: LEADER
I20260810 07:22:07.825564 2810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:07.825544 2816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } }
I20260810 07:22:07.825853 2816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4b70b00cbb4978a35b40731f10eb22. Latest consensus state: current_term: 2 leader_uuid: "1c4b70b00cbb4978a35b40731f10eb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } } }
I20260810 07:22:07.825960 2816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:07.826102 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1c4b70b00cbb4978a35b40731f10eb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4b70b00cbb4978a35b40731f10eb22" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35525 } } }
I20260810 07:22:07.826182 2824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:07.826274 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:07.826769 2824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:07.826930 2824 catalog_manager.cc:1259] Loaded cluster ID: 456c4ed2ca2a414ca724169ce5688d15
I20260810 07:22:07.826979 2824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:07.827462 2810 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:22:07.831354 2824 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:07.832103 2824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Loaded TSK: 0
I20260810 07:22:07.832422 2824 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:22:07.938050 2813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:07.938341 2813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:07.938396 2813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:07.940166 2813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:07.940214 2813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:07.940253 2813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:07.942476 2813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:33343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=37775
--tserver_master_addrs=127.14.164.62:35525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:07.942711 2813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:07.942968 2813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:07.949904 2876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.950085 2874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:07.949931 2873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:07.950699 2813 server_base.cc:1034] running on GCE node
I20260810 07:22:07.950946 2813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:07.951287 2813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:07.952504 2813 hybrid_clock.cc:648] HybridClock initialized: now 1786346527952478 us; error 46 us; skew 500 ppm
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:07.956018 2813 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:07.956601 2813 webserver.cc:460] Webserver started at http://127.14.164.1:37775/ using document root <none> and password file <none>
I20260810 07:22:07.956941 2813 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:07.957055 2813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:07 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:07.978716 2813 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260810 07:22:07.980587 2888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:07.980810 2813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260810 07:22:07.980921 2813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee"
format_stamp: "Formatted at 2026-08-10 07:22:06 on dist-test-slave-x5fp"
server_key: "2b5d5fa9d4b5d2fe2426684d36ab6954"
server_key_iv: "1108fecc1a887a6bb91122c08ceb90a3"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:07.981472 2813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:08.014669 2813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:08.017179 2813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:08.017373 2813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:08.017647 2813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:08.018000 2813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:08.018059 2813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.018107 2813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:08.018137 2813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.024923 2813 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:33343
I20260810 07:22:08.025144 3072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:33343 every 8 connection(s)
I20260810 07:22:08.025445 2813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346527, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:08.035310 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 2813
I20260810 07:22:08.035642 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:44131
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:35525
--webserver_port=42003
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:08.042169 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.026889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:38567 (local address 127.14.164.62:35525)
0810 07:22:08.027652 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:08.027662 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:08.027702 (+ 40us) server_negotiation.cc:407] Connection header received
0810 07:22:08.027855 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:08.027862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:08.027920 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:08.028062 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:08.028900 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.030336 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.031890 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.032541 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.033169 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.035472 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:08.035477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:08.035477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:08.035493 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:08.039761 (+ 4268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.040346 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.040350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.040350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.040488 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.040864 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.040865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.040865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.041205 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:08.041296 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:08.041763 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 07:22:08.041907 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":87,"threads_started":1}
I20260810 07:22:08.042183 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.026476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35525 (local address 127.14.164.1:38567)
0810 07:22:08.026685 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:08.026738 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:08.027416 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:08.028153 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:08.028158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:08.028426 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:08.028745 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:08.028754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.030482 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.030484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:08.031766 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:08.031769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.032679 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.032680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:08.032844 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.033509 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:08.033520 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:08.035278 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:08.039884 (+ 4606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:08.039895 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:08.039897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:08.040231 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:08.040583 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:08.040584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:08.040585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:08.040768 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:08.041317 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:08.041326 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:08.041529 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 07:22:08.041663 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 07:22:08.041865 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260810 07:22:08.043430 3073 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35525
I20260810 07:22:08.043538 3073 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.043751 3073 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
W20260810 07:22:08.044803 3073 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:35525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:22:08.044885 3073 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.044960 3073 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:08.045424 3073 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.045504 3073 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
W20260810 07:22:08.045871 3073 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:22:08.152550 3080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:08.152877 3080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:08.152925 3080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:08.154225 3080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:08.154263 3080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:08.154292 3080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:08.155827 3080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:44131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=42003
--tserver_master_addrs=127.14.164.62:35525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:08.156034 3080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:08.156332 3080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:08.162735 3090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:08.162735 3087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:08.163386 3080 server_base.cc:1034] running on GCE node
W20260810 07:22:08.163628 3086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:08.163868 3080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:08.164149 3080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:08.165320 3080 hybrid_clock.cc:648] HybridClock initialized: now 1786346528165297 us; error 53 us; skew 500 ppm
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:08.168630 3080 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:08.169181 3080 webserver.cc:460] Webserver started at http://127.14.164.2:42003/ using document root <none> and password file <none>
I20260810 07:22:08.169468 3080 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:08.169548 3080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:08.192039 3080 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:08.195077 3104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.195261 3080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:22:08.195343 3080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "615d086687ae4a248c91dacbf348baed"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c761092af420333186e8fe08786d8510"
server_key_iv: "0de5687873961254f4bc835937b02118"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:08.195832 3080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:08.219116 3080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:08.221350 3080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:08.221544 3080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:08.221814 3080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:08.222165 3080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:08.222224 3080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.222266 3080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:08.222292 3080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.229841 3080 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:44131
I20260810 07:22:08.230260 3274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:44131 every 8 connection(s)
I20260810 07:22:08.230338 3080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:08.239214 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 3080
I20260810 07:22:08.239599 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.164.20:41003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:40931
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:35525
--webserver_port=42849
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:08.241218 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.231817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:47517 (local address 127.14.164.62:35525)
0810 07:22:08.231873 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:08.231875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:08.232257 (+ 382us) server_negotiation.cc:407] Connection header received
0810 07:22:08.232990 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:08.232992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:08.233006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:08.233063 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:08.233634 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.234156 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.235295 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.235564 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.235652 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.237902 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:08.237904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:08.237904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:08.237913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:08.239495 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.240122 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.240123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.240123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.240191 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.240471 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.240472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.240473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.240601 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:08.240678 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:08.241022 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 07:22:08.241120 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:22:08.241698 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.231660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35525 (local address 127.14.164.2:47517)
0810 07:22:08.232110 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:08.232156 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:08.232877 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:08.233131 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:08.233136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:08.233324 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:08.233533 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:08.233536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.234265 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.234266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:08.235198 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:08.235200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.235681 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.235683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:08.235821 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.236401 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:08.236410 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:08.237790 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:08.239585 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:08.239593 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:08.239596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:08.240024 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:08.240264 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:08.240264 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:08.240265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:08.240396 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:08.240681 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:08.240689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:08.240879 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 07:22:08.240998 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:22:08.241162 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":143,"threads_started":1}
I20260810 07:22:08.242504 3275 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35525
I20260810 07:22:08.242660 3275 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.242933 3275 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
W20260810 07:22:08.243661 3275 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:35525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:22:08.243753 3275 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.243835 3275 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:08.244143 3275 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.244228 3275 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
W20260810 07:22:08.244519 3275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:22:08.349403 3279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:08.349710 3279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:08.349762 3279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:08.351536 3279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:08.351596 3279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:08.351639 3279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:08.353461 3279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.164.20:41003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:40931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=42849
--tserver_master_addrs=127.14.164.62:35525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:08.353772 3279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:08.354064 3279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:08.359668 3286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:08.359805 3289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:08.359668 3294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:08.361016 3279 server_base.cc:1034] running on GCE node
I20260810 07:22:08.361323 3279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:08.361634 3279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:08.362846 3279 hybrid_clock.cc:648] HybridClock initialized: now 1786346528362835 us; error 31 us; skew 500 ppm
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:08.365919 3279 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:08.366534 3279 webserver.cc:460] Webserver started at http://127.14.164.3:42849/ using document root <none> and password file <none>
I20260810 07:22:08.366822 3279 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:08.366911 3279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:08.405582 3279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:08.406698 3306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.407004 3279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:08.407102 3279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "c2bd92dba909418e86ba2d4a6d75a3d0"
format_stamp: "Formatted at 2026-08-10 07:22:07 on dist-test-slave-x5fp"
server_key: "c6767d91be797082702485fdcd0afeee"
server_key_iv: "64762d7cbde485df5f24e51e6b7ce70b"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:08.407579 3279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:08.433354 3279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:08.435930 3279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:08.436110 3279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:08.436378 3279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:08.436810 3279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:08.436865 3279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.436898 3279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:08.436915 3279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:08.444811 3279 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:40931
I20260810 07:22:08.445261 3279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:08.445822 3481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:40931 every 8 connection(s)
I20260810 07:22:08.451526 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 3279
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:08.461717 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.447200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:45085 (local address 127.14.164.62:35525)
0810 07:22:08.447286 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:08.447288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:08.447472 (+ 184us) server_negotiation.cc:407] Connection header received
0810 07:22:08.447971 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:08.447973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:08.447986 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:08.448071 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:08.448834 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.449587 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.450929 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.451215 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.451323 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.457190 (+ 5867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:08.457191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:08.457192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:08.457200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:08.459049 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.459963 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.459965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.459965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.460031 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.460830 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.460832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.460832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.460987 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:08.461038 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:08.461513 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 07:22:08.461605 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:22:08.462610 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.447148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35525 (local address 127.14.164.3:45085)
0810 07:22:08.447334 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:08.447378 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:08.447863 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:08.448175 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:08.448180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:08.448436 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:08.448685 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:08.448691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.449704 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.449705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:08.450823 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:08.450826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.451382 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.451384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:08.451552 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.452308 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:08.452320 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:08.457062 (+ 4742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:08.459460 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:08.459472 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:08.459475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:08.459869 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:08.460528 (+ 659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:08.460529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:08.460530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:08.460738 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:08.461092 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:08.461099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:08.461316 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 07:22:08.462088 (+ 772us) client_negotiation.cc:240] Negotiation successful
0810 07:22:08.462288 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260810 07:22:08.463442 3483 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35525
I20260810 07:22:08.463574 3483 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.463821 3483 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
W20260810 07:22:08.464614 3483 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:35525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:22:08.464708 3483 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.464830 3483 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:08.465238 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:08.452022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.14.164.62:35525)
0810 07:22:08.452191 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:08.452194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:08.452373 (+ 179us) server_negotiation.cc:407] Connection header received
0810 07:22:08.452521 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:08.452522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:08.452536 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:08.452585 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:08.453518 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.454266 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.456233 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:08.456586 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:08.456658 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:08.459152 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:08.459154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:08.459154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:08.459161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:08.461370 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.463220 (+ 1850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.463221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.463222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.463325 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:08.464406 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:08.464407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:08.464407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:08.464590 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:08.464816 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:08.464994 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 07:22:08.465106 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260810 07:22:08.465260 3483 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:08.465344 3483 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
W20260810 07:22:08.465798 3483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 07:22:08 dist-test-slave-x5fp krb5kdc[1221](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:09.046080 3073 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:09.046270 3073 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:09.046748 2762 ts_manager.cc:194] Registered new tserver with Master: 126b8ed7f1dd4abbb23c974fb0ec2bee (127.14.164.1:33343)
I20260810 07:22:09.047833 2762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:38567
I20260810 07:22:09.244755 3275 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:09.244930 3275 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:09.245294 2762 ts_manager.cc:194] Registered new tserver with Master: 615d086687ae4a248c91dacbf348baed (127.14.164.2:44131)
I20260810 07:22:09.245831 2762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:47517
Aug 10 07:22:09 dist-test-slave-x5fp krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346528, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for HTTP/127.14.164.20@KRBTEST.COM
I20260810 07:22:09.467475 3483 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:09.467638 3483 heartbeater.cc:511] Master 127.14.164.62:35525 requested a full tablet report, sending...
I20260810 07:22:09.468101 2762 ts_manager.cc:194] Registered new tserver with Master: c2bd92dba909418e86ba2d4a6d75a3d0 (127.14.164.3:40931)
I20260810 07:22:09.468636 2762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:45085
I20260810 07:22:09.478096 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:09.492635 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.481440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.14.164.62:35525)
0810 07:22:09.482069 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.482072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.484440 (+ 2368us) server_negotiation.cc:407] Connection header received
0810 07:22:09.484486 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.484487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.484504 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.484590 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:09.485430 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.486241 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.487262 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.487553 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.487600 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.489257 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:09.489259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:09.489259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:09.489268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:09.491246 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:09.491663 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:09.491665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:09.491665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:09.491732 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:09.491959 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:09.491960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:09.491960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:09.492102 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:09.492178 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.492420 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.492531 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":91,"threads_started":1}
I20260810 07:22:09.498062 2762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:09.500248 2762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:09.511431 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.504528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.14.164.1:33343)
0810 07:22:09.504664 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.504669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.505424 (+ 755us) server_negotiation.cc:407] Connection header received
0810 07:22:09.505471 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.505474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.505491 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.505564 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:09.505812 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.507783 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.509852 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.510856 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.510921 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.510985 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.511275 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.511328 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":51,"threads_started":1}
I20260810 07:22:09.512352 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.504253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:33343 (local address 127.0.0.1:47842)
0810 07:22:09.504754 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:09.505120 (+ 366us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:09.505174 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:09.505664 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:09.505667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:09.505670 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:09.505741 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.505743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.507889 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.507890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.509706 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.509725 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.511056 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.511057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.511141 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.511155 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:22:09.512197 (+ 1042us) client_negotiation.cc:240] Negotiation successful
0810 07:22:09.512271 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":98,"threads_started":1}
I20260810 07:22:09.513549 2971 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb401009347457f86740a07ff5d9da2 (DEFAULT_TABLE table=test-table [id=028f0696ca6941db9e459aa8e3a33984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:09.513968 2971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb401009347457f86740a07ff5d9da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:09.515892 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.504648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.14.164.3:40931)
0810 07:22:09.505106 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.505111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.508312 (+ 3201us) server_negotiation.cc:407] Connection header received
0810 07:22:09.508359 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.508362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.508377 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.508444 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:09.508769 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.509777 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.514167 (+ 4390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.514963 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.515041 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.515103 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.515680 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.515736 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":59,"threads_started":1}
I20260810 07:22:09.516470 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.504540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:40931 (local address 127.0.0.1:39560)
0810 07:22:09.507921 (+ 3381us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:09.507932 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:09.507978 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:09.508645 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:09.508646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:09.508648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:09.508670 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.508671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.509874 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.509874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.510930 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.510933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.515144 (+ 4211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.515145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.515210 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.515215 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:22:09.515276 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:22:09.515333 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3362,"thread_start_us":124,"threads_started":1}
I20260810 07:22:09.516973 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.504256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:44131 (local address 127.0.0.1:59432)
0810 07:22:09.504879 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:09.504896 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:09.504964 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:09.508528 (+ 3564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:09.508529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:09.508531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:09.508584 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.508586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.513037 (+ 4451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.513038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.514057 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.514062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.516329 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.516330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.516472 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.516478 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:22:09.516816 (+ 338us) client_negotiation.cc:240] Negotiation successful
0810 07:22:09.516867 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":141,"threads_started":1}
I20260810 07:22:09.517236 3378 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb401009347457f86740a07ff5d9da2 (DEFAULT_TABLE table=test-table [id=028f0696ca6941db9e459aa8e3a33984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:09.517586 3378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb401009347457f86740a07ff5d9da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:09.517709 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.504358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.14.164.2:44131)
0810 07:22:09.508087 (+ 3729us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.508092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.508102 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:09.508137 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.508139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.508156 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.508224 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:09.511766 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.512937 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.515470 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.516243 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.516334 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.516399 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.517270 (+ 871us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.517486 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3691,"thread_start_us":71,"threads_started":1}
I20260810 07:22:09.520596 3530 tablet_bootstrap.cc:492] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Bootstrap starting.
I20260810 07:22:09.521118 3177 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb401009347457f86740a07ff5d9da2 (DEFAULT_TABLE table=test-table [id=028f0696ca6941db9e459aa8e3a33984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:09.521545 3177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb401009347457f86740a07ff5d9da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:09.521566 3530 tablet_bootstrap.cc:654] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:09.522019 3530 log.cc:824] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:09.523270 3530 tablet_bootstrap.cc:492] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: No bootstrap required, opened a new log
I20260810 07:22:09.523363 3530 ts_tablet_manager.cc:1397] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:22:09.526194 3535 tablet_bootstrap.cc:492] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Bootstrap starting.
I20260810 07:22:09.527199 3535 tablet_bootstrap.cc:654] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:09.527551 3530 raft_consensus.cc:348] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.527756 3535 log.cc:824] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:09.527773 3530 raft_consensus.cc:374] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:09.527817 3530 raft_consensus.cc:729] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2bd92dba909418e86ba2d4a6d75a3d0, State: Initialized, Role: FOLLOWER
I20260810 07:22:09.527993 3530 consensus_queue.cc:260] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.528258 3536 tablet_bootstrap.cc:492] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee: Bootstrap starting.
I20260810 07:22:09.528376 3530 ts_tablet_manager.cc:1428] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:22:09.528509 3483 heartbeater.cc:503] Master 127.14.164.62:35525 was elected leader, sending a full tablet report...
I20260810 07:22:09.529273 3536 tablet_bootstrap.cc:654] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:09.529866 3536 log.cc:824] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:09.530922 3535 tablet_bootstrap.cc:492] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: No bootstrap required, opened a new log
I20260810 07:22:09.531008 3535 ts_tablet_manager.cc:1397] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 07:22:09.531242 3536 tablet_bootstrap.cc:492] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee: No bootstrap required, opened a new log
I20260810 07:22:09.531316 3536 ts_tablet_manager.cc:1397] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 07:22:09.534929 3536 raft_consensus.cc:348] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.535148 3536 raft_consensus.cc:374] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:09.535202 3536 raft_consensus.cc:729] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 126b8ed7f1dd4abbb23c974fb0ec2bee, State: Initialized, Role: FOLLOWER
I20260810 07:22:09.535346 3535 raft_consensus.cc:348] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.535439 3536 consensus_queue.cc:260] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.535526 3535 raft_consensus.cc:374] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:09.535573 3535 raft_consensus.cc:729] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615d086687ae4a248c91dacbf348baed, State: Initialized, Role: FOLLOWER
I20260810 07:22:09.535722 3535 consensus_queue.cc:260] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.535844 3536 ts_tablet_manager.cc:1428] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260810 07:22:09.535997 3073 heartbeater.cc:503] Master 127.14.164.62:35525 was elected leader, sending a full tablet report...
I20260810 07:22:09.536019 3535 ts_tablet_manager.cc:1428] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:22:09.536327 3275 heartbeater.cc:503] Master 127.14.164.62:35525 was elected leader, sending a full tablet report...
I20260810 07:22:09.594432 3538 raft_consensus.cc:482] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:09.594633 3538 raft_consensus.cc:504] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.595098 3538 leader_election.cc:288] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 126b8ed7f1dd4abbb23c974fb0ec2bee (127.14.164.1:33343), 615d086687ae4a248c91dacbf348baed (127.14.164.2:44131)
I20260810 07:22:09.600216 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.595387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:39751 (local address 127.14.164.1:33343)
0810 07:22:09.595439 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.595441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.595675 (+ 234us) server_negotiation.cc:407] Connection header received
0810 07:22:09.595788 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.595789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.595798 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.595822 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:09.596171 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.596991 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.599178 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.599735 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.599832 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.599844 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.600088 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.600137 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:22:09.600251 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.595315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:33343 (local address 127.14.164.3:39751)
0810 07:22:09.595594 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:09.595606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:09.595699 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:09.595916 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:09.595917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:09.595919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:09.596043 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.596045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.597094 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.597095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.599030 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.599035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.599935 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.599937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.599998 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.600003 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:22:09.600080 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:22:09.600126 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":6,"thread_start_us":123,"threads_started":1}
I20260810 07:22:09.600764 2999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb401009347457f86740a07ff5d9da2" candidate_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" is_pre_election: true
I20260810 07:22:09.600976 2999 raft_consensus.cc:2454] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2bd92dba909418e86ba2d4a6d75a3d0 in term 0.
I20260810 07:22:09.601251 3309 leader_election.cc:302] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 126b8ed7f1dd4abbb23c974fb0ec2bee, c2bd92dba909418e86ba2d4a6d75a3d0; no voters:
I20260810 07:22:09.601418 3538 raft_consensus.cc:2781] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:09.601500 3538 raft_consensus.cc:482] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:09.601536 3538 raft_consensus.cc:3037] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:09.602273 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.595364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:44131 (local address 127.14.164.3:46459)
0810 07:22:09.595595 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:09.595606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:09.595674 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:09.595919 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:09.595919 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:09.595920 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:09.596141 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.596143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.599205 (+ 3062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.599206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.600881 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:09.600884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.601900 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.601901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:09.602012 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.602017 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:22:09.602083 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:22:09.602124 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
I20260810 07:22:09.602313 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.595588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:46459 (local address 127.14.164.2:44131)
0810 07:22:09.595662 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.595665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.595679 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:22:09.595772 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.595773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.595805 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.595919 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:09.596236 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.596995 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.600980 (+ 3985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.601704 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.601900 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.601919 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.602108 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.602195 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:22:09.602638 3538 raft_consensus.cc:504] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.602998 2999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb401009347457f86740a07ff5d9da2" candidate_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee"
I20260810 07:22:09.603060 3202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb401009347457f86740a07ff5d9da2" candidate_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615d086687ae4a248c91dacbf348baed" is_pre_election: true
I20260810 07:22:09.603123 2999 raft_consensus.cc:3037] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:09.603206 3202 raft_consensus.cc:2454] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2bd92dba909418e86ba2d4a6d75a3d0 in term 0.
I20260810 07:22:09.603394 3538 leader_election.cc:288] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [CANDIDATE]: Term 1 election: Requested vote from peers 126b8ed7f1dd4abbb23c974fb0ec2bee (127.14.164.1:33343), 615d086687ae4a248c91dacbf348baed (127.14.164.2:44131)
I20260810 07:22:09.603588 3202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb401009347457f86740a07ff5d9da2" candidate_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615d086687ae4a248c91dacbf348baed"
I20260810 07:22:09.603677 3202 raft_consensus.cc:3037] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:09.604339 2999 raft_consensus.cc:2454] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2bd92dba909418e86ba2d4a6d75a3d0 in term 1.
I20260810 07:22:09.604578 3309 leader_election.cc:302] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 126b8ed7f1dd4abbb23c974fb0ec2bee, c2bd92dba909418e86ba2d4a6d75a3d0; no voters:
I20260810 07:22:09.604709 3538 raft_consensus.cc:2781] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:09.605018 3538 raft_consensus.cc:686] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 1 LEADER]: Becoming Leader. State: Replica: c2bd92dba909418e86ba2d4a6d75a3d0, State: Running, Role: LEADER
I20260810 07:22:09.605141 3202 raft_consensus.cc:2454] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2bd92dba909418e86ba2d4a6d75a3d0 in term 1.
I20260810 07:22:09.605157 3538 consensus_queue.cc:237] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } }
I20260810 07:22:09.606130 2761 catalog_manager.cc:5495] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2bd92dba909418e86ba2d4a6d75a3d0 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2bd92dba909418e86ba2d4a6d75a3d0" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40931 } health_report { overall_health: HEALTHY } } }
I20260810 07:22:09.627573 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:09.623554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.14.164.3:40931)
0810 07:22:09.623645 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:09.623647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:09.623659 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:22:09.623706 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:09.623707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:09.623739 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:09.623825 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:09.623989 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.624883 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.626000 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:09.626335 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:09.626410 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:09.626601 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:09.626984 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:09.627081 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:09.627444 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 07:22:09.627478 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:22:09.630438 3202 raft_consensus.cc:1261] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 1 FOLLOWER]: Refusing update from remote peer c2bd92dba909418e86ba2d4a6d75a3d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:09.630458 2999 raft_consensus.cc:1261] T 7eb401009347457f86740a07ff5d9da2 P 126b8ed7f1dd4abbb23c974fb0ec2bee [term 1 FOLLOWER]: Refusing update from remote peer c2bd92dba909418e86ba2d4a6d75a3d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:09.630921 3545 consensus_queue.cc:1035] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "126b8ed7f1dd4abbb23c974fb0ec2bee" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:09.631080 3538 consensus_queue.cc:1035] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615d086687ae4a248c91dacbf348baed" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 44131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:09.635699 3553 mvcc.cc:205] Tried to move back new op lower bound from 7316875385362923520 to 7316875385263636480. Current Snapshot: MvccSnapshot[applied={T|T < 7316875385362923520}]
I20260810 07:22:09.638895 2762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:09.639038 2762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:09.640298 2762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4b70b00cbb4978a35b40731f10eb22: Sending DeleteTablet for 3 replicas of tablet 7eb401009347457f86740a07ff5d9da2
I20260810 07:22:09.640974 2971 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb401009347457f86740a07ff5d9da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:09 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:47842
I20260810 07:22:09.641371 3177 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb401009347457f86740a07ff5d9da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:09 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59432
I20260810 07:22:09.641721 3378 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb401009347457f86740a07ff5d9da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:09 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:39560
I20260810 07:22:09.643069 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 2813
I20260810 07:22:09.664491 3565 tablet_replica.cc:330] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: stopping tablet replica
I20260810 07:22:09.664830 3565 raft_consensus.cc:2229] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:22:09.665709 3565 raft_consensus.cc:2258] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:09.666082 3563 tablet_replica.cc:330] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: stopping tablet replica
I20260810 07:22:09.666195 3563 raft_consensus.cc:2229] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:09.666324 3563 raft_consensus.cc:2258] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:09.666683 3565 ts_tablet_manager.cc:1905] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:09.669214 3563 ts_tablet_manager.cc:1905] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:09.669535 3565 ts_tablet_manager.cc:1918] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:09.669693 3565 log.cc:1196] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/7eb401009347457f86740a07ff5d9da2
I20260810 07:22:09.670094 3565 ts_tablet_manager.cc:1939] T 7eb401009347457f86740a07ff5d9da2 P c2bd92dba909418e86ba2d4a6d75a3d0: Deleting consensus metadata
I20260810 07:22:09.670673 2746 catalog_manager.cc:4841] TS c2bd92dba909418e86ba2d4a6d75a3d0 (127.14.164.3:40931): tablet 7eb401009347457f86740a07ff5d9da2 (table test-table [id=028f0696ca6941db9e459aa8e3a33984]) successfully deleted
I20260810 07:22:09.670871 3563 ts_tablet_manager.cc:1918] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:09.670960 3563 log.cc:1196] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/7eb401009347457f86740a07ff5d9da2
I20260810 07:22:09.671229 3563 ts_tablet_manager.cc:1939] T 7eb401009347457f86740a07ff5d9da2 P 615d086687ae4a248c91dacbf348baed: Deleting consensus metadata
I20260810 07:22:09.671697 2746 catalog_manager.cc:4841] TS 615d086687ae4a248c91dacbf348baed (127.14.164.2:44131): tablet 7eb401009347457f86740a07ff5d9da2 (table test-table [id=028f0696ca6941db9e459aa8e3a33984]) successfully deleted
W20260810 07:22:09.672227 2744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:33343 (error 108)
W20260810 07:22:09.680649 3527 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:22:09.672685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:33343 (local address 127.0.0.1:47850)
0810 07:22:09.672766 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:09.672787 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:09.672830 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:09.680565 (+ 7735us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:33343: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 07:22:09.680881 2744 catalog_manager.cc:4584] TS 126b8ed7f1dd4abbb23c974fb0ec2bee (127.14.164.1:33343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7eb401009347457f86740a07ff5d9da2: Network error: Client connection negotiation failed: client connection to 127.14.164.1:33343: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:22:09.681449 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 3080
I20260810 07:22:09.694072 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 3279
I20260810 07:22:09.707196 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 2721
I20260810 07:22:09.723766 14992 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:22:10.074550 14992 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:22:10.074645 14992 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:22:10.429530 14992 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:22:10.429 UTC [1266] LOG: received smart shutdown request
2026-08-10 07:22:10.434 UTC [1266] LOG: background worker "logical replication launcher" (PID 1283) exited with exit code 1
2026-08-10 07:22:10.434 UTC [1275] LOG: shutting down
2026-08-10 07:22:10.617 UTC [1266] LOG: database system is shut down
2026-08-10T07:22:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51993 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:10 dist-test-slave-x5fp krb5kdc[3579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:10 dist-test-slave-x5fp krb5kdc[3579](info): set up 2 sockets
Aug 10 07:22:10 dist-test-slave-x5fp krb5kdc[3579](info): commencing operation
krb5kdc: starting...
W20260810 07:22:12.053934 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:22:12.195905 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42045
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42045,127.14.164.61:40841,127.14.164.60:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:12.281479 3603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:12.281699 3603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:12.281723 3603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:12.282922 3603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:12.282968 3603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:12.282982 3603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:12.282995 3603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:12.283007 3603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:12.284541 3603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42045,127.14.164.61:40841,127.14.164.60:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:12.284834 3603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:12.285080 3603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:12.290270 3616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:12.290351 3613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:12.290252 3612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:12.290475 3603 server_base.cc:1034] running on GCE node
I20260810 07:22:12.290746 3603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:12.291071 3603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:12.292300 3603 hybrid_clock.cc:648] HybridClock initialized: now 1786346532292269 us; error 46 us; skew 500 ppm
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:12.295015 3603 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:12.295495 3603 webserver.cc:460] Webserver started at http://127.14.164.62:43779/ using document root <none> and password file <none>
I20260810 07:22:12.295730 3603 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:12.295802 3603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:12.295959 3603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:12.297197 3603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "8fce15df72504fe288bc5abd3b6de13f"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "ede1810fcbf24ae673b81b56b9f23b84"
server_key_iv: "4f43d6cf2e33668e2cc0526024983664"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.297592 3603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "8fce15df72504fe288bc5abd3b6de13f"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "ede1810fcbf24ae673b81b56b9f23b84"
server_key_iv: "4f43d6cf2e33668e2cc0526024983664"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.299026 3603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:22:12.299834 3628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:12.300022 3603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:22:12.300140 3603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "8fce15df72504fe288bc5abd3b6de13f"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "ede1810fcbf24ae673b81b56b9f23b84"
server_key_iv: "4f43d6cf2e33668e2cc0526024983664"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.300225 3603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:12.318411 3603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:12.320756 3603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:12.320976 3603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:12.325115 3603 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42045
I20260810 07:22:12.325152 3703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42045 every 8 connection(s)
I20260810 07:22:12.325686 3603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:12.326403 3704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:12.328068 3704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:12.332396 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 3603
I20260810 07:22:12.332526 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:22:12.332803 14992 external_mini_cluster.cc:1428] Setting key c7cbab25e1d860cc5992317c93d811ae
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:12.339236 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.328866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42045 (local address 127.0.0.1:43794)
0810 07:22:12.329027 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.329066 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.329540 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.329833 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.329835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.330121 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.330288 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.330292 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.331405 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.331406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.332106 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.332108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.332958 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.332959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.333034 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.333626 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.333636 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.335280 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.337670 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.337677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.337679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.337879 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.338210 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.338211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.338211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.338301 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.338759 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.338764 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.338820 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.338922 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.339062 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260810 07:22:12.339371 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.329033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.14.164.62:42045)
0810 07:22:12.329165 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.329168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.329199 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:22:12.329685 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.329687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.329715 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.329795 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.330407 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.331300 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.332249 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.332830 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.332953 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.335390 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.335398 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.335399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.335412 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.337574 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.338007 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.338011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.338012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.338133 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.338372 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.338373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.338374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.338685 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.338780 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.338985 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.339162 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
I20260810 07:22:12.340157 3704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:12.340557 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.340431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58196)
0810 07:22:12.340484 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.340535 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:22:12.340692 3635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
I20260810 07:22:12.341172 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.341078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58210)
0810 07:22:12.341122 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.341148 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 07:22:12.341271 3704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:40841: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.164.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab.
Entry for principal kudu/127.14.164.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab.
I20260810 07:22:12.365046 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:40841
--webserver_interface=127.14.164.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab
--principal=kudu/127.14.164.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42045,127.14.164.61:40841,127.14.164.60:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:12.390499 3704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } attempt: 1
I20260810 07:22:12.391094 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.390932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58224)
0810 07:22:12.391010 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.391058 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 07:22:12.391471 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.391367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58226)
0810 07:22:12.391415 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.391444 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:22:12.391559 3704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:40841: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
W20260810 07:22:12.451121 3714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:12.451380 3714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:12.451417 3714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:12.452615 3714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:12.452685 3714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:12.452709 3714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:12.452777 3714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:12.452803 3714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:12.454473 3714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42045,127.14.164.61:40841,127.14.164.60:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab
--principal=kudu/127.14.164.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:40841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.164.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:12.454782 3714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:12.455037 3714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:12.459725 3704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } attempt: 2
I20260810 07:22:12.460253 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.460102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58240)
0810 07:22:12.460176 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.460211 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 07:22:12.460328 3721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:12.460604 3714 server_base.cc:1034] running on GCE node
W20260810 07:22:12.460300 3722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:12.460317 3724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:12.460845 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.460679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58246)
0810 07:22:12.460768 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.460817 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 07:22:12.461030 3704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:40841: Network error: Client connection negotiation failed: client connection to 127.14.164.61:40841: connect: Connection refused (error 111)
I20260810 07:22:12.461117 3714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:12.461440 3714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:12.462612 3714 hybrid_clock.cc:648] HybridClock initialized: now 1786346532462582 us; error 43 us; skew 500 ppm
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:12.465567 3714 init.cc:377] Logged in from keytab as kudu/127.14.164.61@KRBTEST.COM (short username kudu)
I20260810 07:22:12.466022 3714 webserver.cc:460] Webserver started at http://127.14.164.61:38165/ using document root <none> and password file <none>
I20260810 07:22:12.466233 3714 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:12.466275 3714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:12.466374 3714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:12.467454 3714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data/instance:
uuid: "4d7848f8b1924af988dd2208bf161a66"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "254830f8c66ade06667482567050e76d"
server_key_iv: "3e6b1512bfa56cea0ae4ff3c5e4dd953"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.467777 3714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal/instance:
uuid: "4d7848f8b1924af988dd2208bf161a66"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "254830f8c66ade06667482567050e76d"
server_key_iv: "3e6b1512bfa56cea0ae4ff3c5e4dd953"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.469301 3714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:22:12.470162 3734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:12.470501 3714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:12.470623 3714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal
uuid: "4d7848f8b1924af988dd2208bf161a66"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "254830f8c66ade06667482567050e76d"
server_key_iv: "3e6b1512bfa56cea0ae4ff3c5e4dd953"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.470710 3714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:12.487021 3714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:12.489671 3714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:12.489852 3714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:12.494163 3714 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.61:40841
I20260810 07:22:12.494230 3828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.61:40841 every 8 connection(s)
I20260810 07:22:12.494611 3714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
I20260810 07:22:12.495599 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:12.497328 3829 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:12.501730 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 3714
I20260810 07:22:12.501854 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-1/wal/instance
I20260810 07:22:12.502045 14992 external_mini_cluster.cc:1428] Setting key 0f621ad2ec40f42c4c5ea87c5a7acd47
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:12.507459 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.498089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.14.164.62:42045)
0810 07:22:12.498158 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.498160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.498396 (+ 236us) server_negotiation.cc:407] Connection header received
0810 07:22:12.498838 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.498839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.498852 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.498915 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.499563 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.500097 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.501250 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.501460 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.501544 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.504236 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.504237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.504238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.504246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.505791 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.506323 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.506324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.506324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.506389 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.506734 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.506736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.506736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.506882 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.506951 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.507307 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.507390 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:22:12.507793 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.498066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42045 (local address 127.0.0.1:43802)
0810 07:22:12.498266 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.498309 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.498757 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.498995 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.498998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.499259 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.499462 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.499466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.500208 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.500209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.501161 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.501164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.501576 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.501578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.501702 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.502303 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.502317 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.504127 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.505885 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.505893 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.505896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.506239 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.506474 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.506475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.506476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.506638 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.506967 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.506975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.507160 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.507280 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.507469 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260810 07:22:12.508514 3829 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.164.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.60@KRBTEST.COM" created.
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:22:12.519658 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.508839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58254)
0810 07:22:12.508894 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.508907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.508959 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.509241 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.509243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.509420 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.509497 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.509500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.510412 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.510414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.511243 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.511246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.511732 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.511733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.511822 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.512260 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.512266 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.516498 (+ 4232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.518516 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.518517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.518517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.518692 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.518924 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.518925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.518926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.518990 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.519304 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.519305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.519346 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.519416 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.519535 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:22:12.519702 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.508925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.14.164.61:40841)
0810 07:22:12.509087 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.509092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.509115 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:22:12.509132 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.509133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.509143 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.509200 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.509586 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.510312 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.511338 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.511646 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.511707 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.516613 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.516626 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.516626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.516634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.518395 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.518792 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.518795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.518796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.518857 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.519055 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.519056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.519057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.519227 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.519316 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.519455 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.519593 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260810 07:22:12.520325 3829 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:12.520888 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.520770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40692)
0810 07:22:12.520807 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.520860 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260810 07:22:12.521070 3741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
I20260810 07:22:12.521662 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.521532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40702)
0810 07:22:12.521608 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.521639 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 07:22:12.521816 3829 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:43609: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab.
Entry for principal kudu/127.14.164.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab.
I20260810 07:22:12.536067 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:43609
--webserver_interface=127.14.164.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab
--principal=kudu/127.14.164.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42045,127.14.164.61:40841,127.14.164.60:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:12.552241 3704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } attempt: 3
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:22:12.560405 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.552870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.14.164.61:40841)
0810 07:22:12.552946 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.552948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.552962 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:22:12.553041 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.553042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.553056 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.553130 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.553507 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.554083 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.554797 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.555021 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.555120 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.557290 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.557293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.557293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.557303 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.559142 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.559562 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.559564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.559564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.559632 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.559833 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.559834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.559835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.559965 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.560042 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.560214 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.560300 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:22:12.560405 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.552812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58256)
0810 07:22:12.552867 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.552880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.552969 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.553152 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.553153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.553338 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.553415 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.553418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.554188 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.554190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.554709 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.554711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.555119 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.555121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.555235 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.555772 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.555778 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.557185 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.559295 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.559297 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.559297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.559486 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.559703 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.559704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.559705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.559770 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.560034 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.560035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.560098 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.560173 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.560298 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 07:22:12.560801 3704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:12.561254 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.561136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40710)
0810 07:22:12.561193 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.561231 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:22:12.561604 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.561514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40722)
0810 07:22:12.561559 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.561583 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 07:22:12.561694 3704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:43609: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
I20260810 07:22:12.571228 3829 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } attempt: 1
I20260810 07:22:12.571812 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.571646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40724)
0810 07:22:12.571729 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.571782 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:22:12.572187 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.572101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40738)
0810 07:22:12.572145 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.572170 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:22:12.572321 3829 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:43609: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
I20260810 07:22:12.592856 3704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } attempt: 1
I20260810 07:22:12.593349 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.593206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40742)
0810 07:22:12.593276 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.593318 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:22:12.593677 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.593580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40758)
0810 07:22:12.593627 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.593654 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 07:22:12.593770 3704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:43609: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
W20260810 07:22:12.626475 3840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:12.626734 3840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:12.626772 3840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:12.627960 3840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:12.628031 3840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:12.628055 3840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:12.628075 3840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:12.628106 3840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:12.629724 3840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42045,127.14.164.61:40841,127.14.164.60:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab
--principal=kudu/127.14.164.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.164.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:12.630025 3840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:12.630275 3840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:12.635455 3846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:12.635423 3853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:12.635478 3848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:12.635701 3840 server_base.cc:1034] running on GCE node
I20260810 07:22:12.636268 3840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:12.636660 3840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:12.637897 3840 hybrid_clock.cc:648] HybridClock initialized: now 1786346532637872 us; error 42 us; skew 500 ppm
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:12.640473 3829 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } attempt: 2
I20260810 07:22:12.640820 3840 init.cc:377] Logged in from keytab as kudu/127.14.164.60@KRBTEST.COM (short username kudu)
I20260810 07:22:12.641028 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.640857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40770)
0810 07:22:12.640933 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.640980 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:22:12.641383 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.641279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40778)
0810 07:22:12.641317 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.641357 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 07:22:12.641475 3840 webserver.cc:460] Webserver started at http://127.14.164.60:41169/ using document root <none> and password file <none>
W20260810 07:22:12.641522 3829 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:43609: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
I20260810 07:22:12.641729 3840 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:12.641779 3840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:12.641878 3840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:12.642959 3840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data/instance:
uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "dec7ef2a259ff03a66f66ec42a706403"
server_key_iv: "d9357979480576bdaeb5f6efc5625564"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.643270 3840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal/instance:
uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "dec7ef2a259ff03a66f66ec42a706403"
server_key_iv: "d9357979480576bdaeb5f6efc5625564"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.644841 3840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:22:12.645658 3867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:12.645880 3840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:12.646027 3840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal
uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a"
format_stamp: "Formatted at 2026-08-10 07:22:12 on dist-test-slave-x5fp"
server_key: "dec7ef2a259ff03a66f66ec42a706403"
server_key_iv: "d9357979480576bdaeb5f6efc5625564"
server_key_version: "encryptionkey@0"
I20260810 07:22:12.646126 3840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:12.668941 3704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } attempt: 2
I20260810 07:22:12.669484 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.669326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40784)
0810 07:22:12.669406 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.669450 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:22:12.669839 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.669738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40798)
0810 07:22:12.669787 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.669820 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:22:12.669937 3704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:43609: Network error: Client connection negotiation failed: client connection to 127.14.164.60:43609: connect: Connection refused (error 111)
I20260810 07:22:12.693140 3840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:12.695484 3840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:12.695680 3840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:12.700145 3840 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.60:43609
I20260810 07:22:12.700182 3940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.60:43609 every 8 connection(s)
I20260810 07:22:12.700569 3840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
I20260810 07:22:12.701608 3942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:12.703424 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 3840
I20260810 07:22:12.703526 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346363752364-14992-0/minicluster-data/master-2/wal/instance
I20260810 07:22:12.703732 14992 external_mini_cluster.cc:1428] Setting key f4edc5000fb5da104cdc44ee005a4e29
I20260810 07:22:12.703464 3942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:22:12.714129 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.704463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.14.164.62:42045)
0810 07:22:12.704550 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.704552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.704803 (+ 251us) server_negotiation.cc:407] Connection header received
0810 07:22:12.705414 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.705415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.705429 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.705512 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.706082 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.706661 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.707753 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.708066 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.708156 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.710758 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.710760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.710761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.710769 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.712471 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.713047 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.713048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.713049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.713118 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.713411 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.713412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.713413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.713549 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.713628 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.713926 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.714033 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:22:12.714344 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.704423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42045 (local address 127.0.0.1:43812)
0810 07:22:12.704648 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.704685 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.705279 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.705550 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.705552 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.705814 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.705973 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.705977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.706744 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.706745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.707652 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.707654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.708189 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.708190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.708336 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.709047 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.709059 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.710660 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.712581 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.712589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.712592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.712964 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.713190 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.713190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.713191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.713317 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.713620 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.713628 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.713792 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.713891 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.714059 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":3,"thread_start_us":128,"threads_started":1}
I20260810 07:22:12.714607 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.704675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.14.164.60:43609)
0810 07:22:12.704835 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.704840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.704871 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:22:12.705284 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.705293 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.705321 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.705413 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.706224 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.707258 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.708175 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.708613 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.708706 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.711241 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.711250 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.711250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.711257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.712868 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.713229 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.713233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.713233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.713319 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.713570 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.713571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.713571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.713800 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.713965 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.714052 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.714134 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
I20260810 07:22:12.715051 3942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:12.722985 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.715351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:40841 (local address 127.0.0.1:58268)
0810 07:22:12.715387 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.715398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.715435 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.715654 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.715655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.715820 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.715884 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.715887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.716859 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.716860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.717612 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.717615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.718109 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.718110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.718170 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.718580 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.718584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.719931 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.721934 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.721935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.721936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.722155 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.722385 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.722386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.722386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.722455 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.722737 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.722738 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.722779 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.722832 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.722914 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 07:22:12.723083 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.715434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.14.164.61:40841)
0810 07:22:12.715506 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.715509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.715517 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:22:12.715541 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.715542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.715560 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.715633 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.716012 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.716709 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.717723 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.718027 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.718108 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.720024 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.720025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.720026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.720035 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.721845 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.722244 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.722245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.722245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.722317 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.722525 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.722526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.722526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.722663 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.722735 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.722881 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.722967 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:22:12.723768 3942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:12.724086 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.715561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.14.164.62:42045)
0810 07:22:12.715621 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.715623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.715632 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:22:12.715685 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.715686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.715695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.715744 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.716304 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.716985 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.717882 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.718170 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.718233 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.720810 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.720811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.720811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.720818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.722588 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.723035 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.723037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.723037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.723099 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.723389 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.723390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.723390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.723552 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.723765 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.723806 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.723880 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:22:12.731571 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.724076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40812)
0810 07:22:12.724390 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.724403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.724463 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.724593 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.724594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.724800 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.724891 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.724893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.725843 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.725844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.726544 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.726546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.727000 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.727001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.727038 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.727458 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.727464 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.728663 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.730497 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.730498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.730498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.730689 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.730921 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.730922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.730922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.730994 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.731282 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.731283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.731327 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.731381 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.731459 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295}
I20260810 07:22:12.731673 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.724295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.14.164.60:43609)
0810 07:22:12.724353 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.724355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.724463 (+ 108us) server_negotiation.cc:407] Connection header received
0810 07:22:12.724528 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.724529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.724539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.724578 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.724981 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.725731 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.726617 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.726926 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.726997 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.728804 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.728805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.728806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.728813 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.730416 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.730766 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.730768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.730768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.730844 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.731063 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.731064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.731065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.731211 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.731284 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.731446 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.731531 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:22:12.732676 3829 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } attempt: 3
I20260810 07:22:12.733579 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.724857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.14.164.61:40841)
0810 07:22:12.724917 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.724919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.724929 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:12.725005 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.725006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.725015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.725062 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.725616 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.726291 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.727135 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.727406 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.727475 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.729994 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.729995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.729996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.730003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.731975 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.732380 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.732380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.732381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.732442 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.732676 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.732677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.732677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.732857 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.733370 (+ 513us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.733412 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.733486 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:22:12.733724 3942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: Bootstrap starting.
I20260810 07:22:12.734890 3942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:12.735235 3942 log.cc:824] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:12.735463 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:22:12.736539 3942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: No bootstrap required, opened a new log
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:22:12.739233 3942 raft_consensus.cc:348] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.739418 3942 raft_consensus.cc:374] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:12.739763 3942 raft_consensus.cc:729] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d15daa3c4edb42e0b52ce4d69cecbc5a, State: Initialized, Role: FOLLOWER
I20260810 07:22:12.739916 3942 consensus_queue.cc:260] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.740240 3942 sys_catalog.cc:564] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:12.741894 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.732948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40816)
0810 07:22:12.733031 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.733042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.733085 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.733231 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.733232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.733410 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.733453 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.733455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.734317 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.734319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.735095 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.735099 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.735630 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.735631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.735693 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.736157 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.736164 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.737665 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.740765 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.740766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.740767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.741010 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.741220 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.741221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.741221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.741278 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.741575 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.741576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.741618 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.741666 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.741773 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 07:22:12.742655 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.732989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.14.164.60:43609)
0810 07:22:12.733031 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.733033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.733071 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:22:12.733140 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.733141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.733151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.733525 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.733555 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.734216 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.735201 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.735497 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.735628 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.737751 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.737753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.737753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.737759 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.740648 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.741093 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.741094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.741094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.741153 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.741341 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.741342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.741342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.741500 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.742435 (+ 935us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.742482 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.742567 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:22:12.745028 3829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66: Bootstrap starting.
I20260810 07:22:12.745577 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.736949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.14.164.62:42045)
0810 07:22:12.736994 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.736996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.737251 (+ 255us) server_negotiation.cc:407] Connection header received
0810 07:22:12.737362 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.737363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.737376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.737407 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.738238 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.739017 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.740117 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.740413 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.740485 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.742599 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.742600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.742600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.742608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.744230 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.744653 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.744655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.744655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.744736 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.744968 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.744969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.744969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.745101 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.745175 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.745388 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.745483 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:22:12.745874 3829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:12.746220 3829 log.cc:824] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:12.746345 3960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.736942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.14.164.60:43609)
0810 07:22:12.737797 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.737799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.737819 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:22:12.737837 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.737838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.737857 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.737919 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.738665 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.739399 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.740376 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.740703 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.740853 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.742485 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.742487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.742487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.742494 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.744792 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.745262 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.745263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.745264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.745337 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.745528 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.745529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.745529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.745661 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.745758 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.746133 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.746222 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":84,"threads_started":1}
I20260810 07:22:12.743310 3840 master_runner.cc:186] Error getting master registration for 127.14.164.62:42045: OK, instance_id { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" instance_seqno: 1786346532323885 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:22:12.746934 3840 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:22:12.741283 3965 sys_catalog.cc:455] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.747179 3965 sys_catalog.cc:458] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:22:12.747641 3981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:22:12.747701 3981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:12.748373 3829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66: No bootstrap required, opened a new log
I20260810 07:22:12.750515 3829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.750665 3829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:12.750708 3829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7848f8b1924af988dd2208bf161a66, State: Initialized, Role: FOLLOWER
I20260810 07:22:12.750850 3829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.751199 3829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:12.751157 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.751368 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:12.751549 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.737277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.14.164.61:40841)
0810 07:22:12.737315 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.737317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.738326 (+ 1009us) server_negotiation.cc:407] Connection header received
0810 07:22:12.738413 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.738414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.738424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.738452 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.740949 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.741632 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.746931 (+ 5299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.747213 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.747261 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.748782 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.748784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.748784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.748791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.750249 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.750615 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.750616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.750617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.750700 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.750898 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.750898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.750899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.751030 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.751116 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.751335 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.751464 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 07:22:12.753314 4001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:22:12.753428 4001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:12.753474 3714 master_runner.cc:186] Error getting master registration for 127.14.164.62:42045: OK, instance_id { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" instance_seqno: 1786346532323885 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:22:12.755192 3714 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:22:12.769097 3704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } attempt: 3
Aug 10 07:22:12 dist-test-slave-x5fp krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:22:12.776830 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.769546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.14.164.60:43609)
0810 07:22:12.769590 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:12.769592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:12.769601 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:22:12.769626 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:12.769627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:12.769637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:12.769708 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:12.770068 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.770550 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.771542 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.771820 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.771884 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.774100 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:12.774102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:12.774102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:12.774112 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:12.775585 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.775976 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.775977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.775978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.776037 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:12.776255 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:12.776256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:12.776256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:12.776374 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:12.776449 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:12.776624 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 07:22:12.776709 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:22:12.776830 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:12.769452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:43609 (local address 127.0.0.1:40832)
0810 07:22:12.769506 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:12.769520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:12.769568 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:12.769730 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:12.769731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:12.769903 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:12.769986 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.769988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.770643 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.770644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.771253 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:12.771257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:12.771902 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:12.771903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:12.771970 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:12.772479 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:12.772485 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:12.774012 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:12.775691 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.775693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.775693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.775877 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.776113 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:12.776114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:12.776114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:12.776184 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:12.776453 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:12.776454 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:12.776502 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:22:12.776584 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:22:12.776692 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:22:12.778591 3704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: Bootstrap starting.
I20260810 07:22:12.779284 3704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:12.779618 3704 log.cc:824] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:12.780488 3704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: No bootstrap required, opened a new log
I20260810 07:22:12.782186 3704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.782344 3704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:12.782393 3704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fce15df72504fe288bc5abd3b6de13f, State: Initialized, Role: FOLLOWER
I20260810 07:22:12.782536 3704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.782864 4006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.782927 3704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:12.783015 4006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:22:12.784941 4020 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:22:12.785050 4020 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:12.787925 3603 master_runner.cc:263] No leader master found from master 8fce15df72504fe288bc5abd3b6de13f
I20260810 07:22:12.787976 3603 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:22:12.821959 3639 tablet.cc:2369] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:12.828311 3985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:12.828461 3985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.829005 3985 leader_election.cc:288] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fce15df72504fe288bc5abd3b6de13f (127.14.164.62:42045), d15daa3c4edb42e0b52ce4d69cecbc5a (127.14.164.60:43609)
I20260810 07:22:12.829509 3671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d7848f8b1924af988dd2208bf161a66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fce15df72504fe288bc5abd3b6de13f" is_pre_election: true
I20260810 07:22:12.829675 3671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d7848f8b1924af988dd2208bf161a66 in term 0.
I20260810 07:22:12.830129 3740 leader_election.cc:302] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d7848f8b1924af988dd2208bf161a66, 8fce15df72504fe288bc5abd3b6de13f; no voters:
I20260810 07:22:12.830629 3909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d7848f8b1924af988dd2208bf161a66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" is_pre_election: true
I20260810 07:22:12.830807 3909 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d7848f8b1924af988dd2208bf161a66 in term 0.
I20260810 07:22:12.831138 3985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:12.831210 3985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:12.831238 3985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:12.832332 3985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.832518 3985 leader_election.cc:288] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [CANDIDATE]: Term 1 election: Requested vote from peers 8fce15df72504fe288bc5abd3b6de13f (127.14.164.62:42045), d15daa3c4edb42e0b52ce4d69cecbc5a (127.14.164.60:43609)
I20260810 07:22:12.833050 3671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d7848f8b1924af988dd2208bf161a66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fce15df72504fe288bc5abd3b6de13f"
I20260810 07:22:12.833142 3671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:12.834158 3671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d7848f8b1924af988dd2208bf161a66 in term 1.
I20260810 07:22:12.834421 3740 leader_election.cc:302] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d7848f8b1924af988dd2208bf161a66, 8fce15df72504fe288bc5abd3b6de13f; no voters:
I20260810 07:22:12.834712 3909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d7848f8b1924af988dd2208bf161a66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a"
I20260810 07:22:12.834810 3909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:12.835711 3909 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d7848f8b1924af988dd2208bf161a66 in term 1.
I20260810 07:22:12.837085 3985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:12.837261 3985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [term 1 LEADER]: Becoming Leader. State: Replica: 4d7848f8b1924af988dd2208bf161a66, State: Running, Role: LEADER
I20260810 07:22:12.837380 3985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } }
I20260810 07:22:12.837822 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d7848f8b1924af988dd2208bf161a66. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.837898 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:12.838249 4025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:12.838498 4025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:12.840796 3909 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [term 1 FOLLOWER]: Refusing update from remote peer 4d7848f8b1924af988dd2208bf161a66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:12.841109 3671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [term 1 FOLLOWER]: Refusing update from remote peer 4d7848f8b1924af988dd2208bf161a66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:12.841663 3985 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:12.841840 3985 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:12.843348 4006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d7848f8b1924af988dd2208bf161a66. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.843448 4006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:12.844588 3965 sys_catalog.cc:455] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d7848f8b1924af988dd2208bf161a66. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.844688 3965 sys_catalog.cc:458] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:12.847193 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.847285 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:12.847870 3965 sys_catalog.cc:455] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.847962 3965 sys_catalog.cc:458] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:12.848250 4026 mvcc.cc:205] Tried to move back new op lower bound from 7316875398512615424 to 7316875398502469632. Current Snapshot: MvccSnapshot[applied={T|T < 7316875398512615424}]
I20260810 07:22:12.849959 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.850065 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:12.850656 4006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d7848f8b1924af988dd2208bf161a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fce15df72504fe288bc5abd3b6de13f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42045 } } peers { permanent_uuid: "4d7848f8b1924af988dd2208bf161a66" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 40841 } } peers { permanent_uuid: "d15daa3c4edb42e0b52ce4d69cecbc5a" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 43609 } } }
I20260810 07:22:12.850744 4006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:12.851191 4032 mvcc.cc:205] Tried to move back new op lower bound from 7316875398512615424 to 7316875398502469632. Current Snapshot: MvccSnapshot[applied={T|T < 7316875398512615424}]
I20260810 07:22:12.851341 4030 mvcc.cc:205] Tried to move back new op lower bound from 7316875398512615424 to 7316875398502469632. Current Snapshot: MvccSnapshot[applied={T|T < 7316875398512615424}]
I20260810 07:22:12.853032 4025 catalog_manager.cc:1347] Generated new cluster ID: 45cab09bcb544dbc8708c4c97b28973f
I20260810 07:22:12.853085 4025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:12.883695 4025 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:12.884930 4025 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:12.912640 4025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d7848f8b1924af988dd2208bf161a66: Generated new TSK 0
I20260810 07:22:12.913043 4025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:22:13.748185 3981 catalog_manager.cc:1259] Loaded cluster ID: 45cab09bcb544dbc8708c4c97b28973f
I20260810 07:22:13.748347 3981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: loading cluster ID for follower catalog manager: success
I20260810 07:22:13.749321 3981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: acquiring CA information for follower catalog manager: success
I20260810 07:22:13.749651 3981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d15daa3c4edb42e0b52ce4d69cecbc5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:22:13.785576 4020 catalog_manager.cc:1259] Loaded cluster ID: 45cab09bcb544dbc8708c4c97b28973f
I20260810 07:22:13.785665 4020 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: loading cluster ID for follower catalog manager: success
I20260810 07:22:13.786542 4020 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: acquiring CA information for follower catalog manager: success
I20260810 07:22:13.786835 4020 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fce15df72504fe288bc5abd3b6de13f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:22:13.972002 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 3603
I20260810 07:22:13.985433 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 3714
I20260810 07:22:14.001276 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 3840
2026-08-10T07:22:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3276 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:14 dist-test-slave-x5fp krb5kdc[4044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:14 dist-test-slave-x5fp krb5kdc[4044](info): set up 2 sockets
Aug 10 07:22:14 dist-test-slave-x5fp krb5kdc[4044](info): commencing operation
krb5kdc: starting...
W20260810 07:22:15.296947 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:22:15 dist-test-slave-x5fp krb5kdc[4044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:22:15.431818 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42603
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:15.518536 4064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:15.518819 4064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:15.518874 4064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:15.520136 4064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:15.520216 4064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:15.520241 4064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:15.520262 4064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:15.520295 4064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:15.521958 4064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:15.522285 4064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:15.522550 4064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:15.528064 4064 server_base.cc:1034] running on GCE node
W20260810 07:22:15.528012 4074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:15.528013 4078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:15.528029 4075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:15.528522 4064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:15.528936 4064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:15.530086 4064 hybrid_clock.cc:648] HybridClock initialized: now 1786346535530065 us; error 45 us; skew 500 ppm
Aug 10 07:22:15 dist-test-slave-x5fp krb5kdc[4044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346535, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:15.532996 4064 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:15.533556 4064 webserver.cc:460] Webserver started at http://127.14.164.62:43345/ using document root <none> and password file <none>
I20260810 07:22:15.533820 4064 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:15.533891 4064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:15.534044 4064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:15.535152 4064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "0f525afbfbe34d58879ddbed3f4943b5"
format_stamp: "Formatted at 2026-08-10 07:22:15 on dist-test-slave-x5fp"
server_key: "57cc8397ea0e728b1e5da0b72affa398"
server_key_iv: "3812d4525dcd2cc524741cdc2d501931"
server_key_version: "encryptionkey@0"
I20260810 07:22:15.535633 4064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "0f525afbfbe34d58879ddbed3f4943b5"
format_stamp: "Formatted at 2026-08-10 07:22:15 on dist-test-slave-x5fp"
server_key: "57cc8397ea0e728b1e5da0b72affa398"
server_key_iv: "3812d4525dcd2cc524741cdc2d501931"
server_key_version: "encryptionkey@0"
I20260810 07:22:15.537163 4064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260810 07:22:15.538115 4092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:15.538311 4064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:15.538426 4064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "0f525afbfbe34d58879ddbed3f4943b5"
format_stamp: "Formatted at 2026-08-10 07:22:15 on dist-test-slave-x5fp"
server_key: "57cc8397ea0e728b1e5da0b72affa398"
server_key_iv: "3812d4525dcd2cc524741cdc2d501931"
server_key_version: "encryptionkey@0"
I20260810 07:22:15.538519 4064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:15.550626 4064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:15.553431 4064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:15.553614 4064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:15.558018 4064 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42603
I20260810 07:22:15.558066 4181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42603 every 8 connection(s)
I20260810 07:22:15.558485 4064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:15.559459 4185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:15.562047 4185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5: Bootstrap starting.
I20260810 07:22:15.562628 4185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:15.562899 4185 log.cc:824] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:15.563645 4185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5: No bootstrap required, opened a new log
I20260810 07:22:15.565389 4185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f525afbfbe34d58879ddbed3f4943b5" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42603 } }
I20260810 07:22:15.565543 4185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:15.565567 4185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f525afbfbe34d58879ddbed3f4943b5, State: Initialized, Role: FOLLOWER
I20260810 07:22:15.565716 4185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f525afbfbe34d58879ddbed3f4943b5" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42603 } }
I20260810 07:22:15.565804 4185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:15.565827 4185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:15.565860 4185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:15.566426 4185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f525afbfbe34d58879ddbed3f4943b5" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42603 } }
I20260810 07:22:15.566519 4185 leader_election.cc:302] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f525afbfbe34d58879ddbed3f4943b5; no voters:
I20260810 07:22:15.566653 4185 leader_election.cc:288] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:15.566727 4189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:15.566964 4189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [term 1 LEADER]: Becoming Leader. State: Replica: 0f525afbfbe34d58879ddbed3f4943b5, State: Running, Role: LEADER
I20260810 07:22:15.566946 4185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:15.567111 4189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f525afbfbe34d58879ddbed3f4943b5" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42603 } }
I20260810 07:22:15.567586 4190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f525afbfbe34d58879ddbed3f4943b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f525afbfbe34d58879ddbed3f4943b5" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42603 } } }
I20260810 07:22:15.567674 4190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:15.567870 4191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f525afbfbe34d58879ddbed3f4943b5. Latest consensus state: current_term: 1 leader_uuid: "0f525afbfbe34d58879ddbed3f4943b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f525afbfbe34d58879ddbed3f4943b5" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42603 } } }
I20260810 07:22:15.567981 4191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:15.568292 4203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:15.568656 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 4064
I20260810 07:22:15.568819 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:22:15.568907 4203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:15.569046 14992 external_mini_cluster.cc:1428] Setting key 7de6a9bdc02458a134778a9d00d589b2
I20260810 07:22:15.571241 4203 catalog_manager.cc:1347] Generated new cluster ID: 310519309ce647d193528f927e41c4ff
I20260810 07:22:15.571293 4203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:22:15 dist-test-slave-x5fp krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:15.577251 4203 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:15.577932 4203 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:15.580459 4203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f525afbfbe34d58879ddbed3f4943b5: Generated new TSK 0
I20260810 07:22:15.580765 4203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:22:15.581663 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:15.569712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.14.164.62:42603)
0810 07:22:15.569884 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:15.569893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:15.569925 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:22:15.570048 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:15.570055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:15.570109 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:15.570261 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:15.570911 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:15.572328 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:15.573504 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:15.574180 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:15.574354 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:15.577421 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:15.577425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:15.577426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:15.577439 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:15.579613 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:15.580072 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:15.580075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:15.580076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:15.580215 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:15.580467 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:15.580468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:15.580469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:15.580923 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:15.581054 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:15.581237 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 07:22:15.581368 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260810 07:22:15.582751 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:22:15.585860 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 4064
2026-08-10T07:22:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1582 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:15 dist-test-slave-x5fp krb5kdc[4223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:15 dist-test-slave-x5fp krb5kdc[4223](info): set up 2 sockets
Aug 10 07:22:15 dist-test-slave-x5fp krb5kdc[4223](info): commencing operation
krb5kdc: starting...
W20260810 07:22:16.871608 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:22:16 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:22:17.015830 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35471
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:17.101202 4253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:17.101472 4253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:17.101513 4253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:17.102710 4253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:17.102793 4253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:17.102818 4253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:17.102850 4253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:17.102897 4253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:17.104439 4253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:17.104797 4253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:17.105052 4253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:17.110263 4258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.110287 4260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.110257 4262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:17.110670 4253 server_base.cc:1034] running on GCE node
I20260810 07:22:17.110949 4253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:17.111263 4253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:17.112442 4253 hybrid_clock.cc:648] HybridClock initialized: now 1786346537112406 us; error 40 us; skew 500 ppm
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:17.119565 4253 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:17.120134 4253 webserver.cc:460] Webserver started at http://127.14.164.62:38499/ using document root <none> and password file <none>
I20260810 07:22:17.120411 4253 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:17.120461 4253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:17.120622 4253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:17.121848 4253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "76cd9be632a4280f974d26adad6c9f2e"
server_key_iv: "520eaa813d9de19cadc132f321db4b2d"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.122284 4253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "76cd9be632a4280f974d26adad6c9f2e"
server_key_iv: "520eaa813d9de19cadc132f321db4b2d"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.123746 4253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:22:17.124831 4276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.125061 4253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:17.125168 4253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "76cd9be632a4280f974d26adad6c9f2e"
server_key_iv: "520eaa813d9de19cadc132f321db4b2d"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.125232 4253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:17.132879 4253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:17.135269 4253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:17.135388 4253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:17.139884 4253 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:35471
I20260810 07:22:17.139945 4361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:35471 every 8 connection(s)
I20260810 07:22:17.140316 4253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:17.141520 4362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:17.142534 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 4253
I20260810 07:22:17.142665 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:22:17.143330 14992 external_mini_cluster.cc:1428] Setting key 5ce7b1cc188e0225bd670c878746b504
I20260810 07:22:17.146128 4362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap starting.
I20260810 07:22:17.147197 4362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:17.147629 4362 log.cc:824] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:17.148599 4362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: No bootstrap required, opened a new log
I20260810 07:22:17.151034 4362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:17.151206 4362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:17.151232 4362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0bd29f032548be8dc8ac7eb6ff5f33, State: Initialized, Role: FOLLOWER
I20260810 07:22:17.151400 4362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:17.151486 4362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:17.151510 4362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:17.151582 4362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:17.152465 4362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:17.152609 4362 leader_election.cc:302] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a0bd29f032548be8dc8ac7eb6ff5f33; no voters:
I20260810 07:22:17.152878 4362 leader_election.cc:288] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:17.152915 4368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:17.153048 4368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 LEADER]: Becoming Leader. State: Replica: 4a0bd29f032548be8dc8ac7eb6ff5f33, State: Running, Role: LEADER
I20260810 07:22:17.153204 4368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:17.153209 4362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:17.153735 4374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a0bd29f032548be8dc8ac7eb6ff5f33. Latest consensus state: current_term: 1 leader_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } } }
I20260810 07:22:17.153818 4374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:17.153981 4370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } } }
I20260810 07:22:17.154047 4370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:17.154191 4378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:17.154982 4378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:17.157267 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.144029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.14.164.62:35471)
0810 07:22:17.144331 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.144353 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.144403 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:22:17.145099 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.145110 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.145183 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.145356 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:17.146260 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.148025 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.148973 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.149601 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.149736 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.152657 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:17.152664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:17.152664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:17.152678 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:17.155295 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.155721 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.155726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.155726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.155854 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.156142 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.156143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.156144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.156524 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:17.156694 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.156881 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.157050 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
I20260810 07:22:17.157378 4378 catalog_manager.cc:1347] Generated new cluster ID: b8834e0ed5714912bc5b3a15f49d94d4
I20260810 07:22:17.157418 4378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:17.175753 4378 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:17.176434 4378 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:17.185251 4378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Generated new TSK 0
I20260810 07:22:17.185513 4378 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:22:17.225666 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35471
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:17.315791 4401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:17.316021 4401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:17.316047 4401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:17.317245 4401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:17.317294 4401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:17.317327 4401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:17.318763 4401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:17.318964 4401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:17.319162 4401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:17.324765 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.324769 4408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.324770 4412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:17.325079 4401 server_base.cc:1034] running on GCE node
I20260810 07:22:17.325361 4401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:17.325742 4401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:17.326910 4401 hybrid_clock.cc:648] HybridClock initialized: now 1786346537326902 us; error 48 us; skew 500 ppm
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:17.329998 4401 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:17.330571 4401 webserver.cc:460] Webserver started at http://127.14.164.1:42699/ using document root <none> and password file <none>
I20260810 07:22:17.330843 4401 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:17.330922 4401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:17.331092 4401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:17.332299 4401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "6abbce2e828b46e9a78ec38bc532dc86"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "37021986e815b7a35343af958facfaef"
server_key_iv: "71bea22e951f0dbd62909bff02acbe7b"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.332793 4401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "6abbce2e828b46e9a78ec38bc532dc86"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "37021986e815b7a35343af958facfaef"
server_key_iv: "71bea22e951f0dbd62909bff02acbe7b"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.334306 4401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:17.335176 4420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.335372 4401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:17.335489 4401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "6abbce2e828b46e9a78ec38bc532dc86"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "37021986e815b7a35343af958facfaef"
server_key_iv: "71bea22e951f0dbd62909bff02acbe7b"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.335584 4401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:17.344573 4401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:17.346797 4401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:17.346989 4401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:17.347265 4401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:17.347616 4401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:17.347681 4401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.347741 4401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:17.347769 4401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.355095 4401 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:34753
I20260810 07:22:17.355136 4599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:34753 every 8 connection(s)
I20260810 07:22:17.355583 4401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:17.362576 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 4401
I20260810 07:22:17.362701 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:22:17.362895 14992 external_mini_cluster.cc:1428] Setting key 1d2833acc23f9d89796985bfa586d0c5
I20260810 07:22:17.366993 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.356686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:40209 (local address 127.14.164.62:35471)
0810 07:22:17.356917 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.356919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.356980 (+ 61us) server_negotiation.cc:407] Connection header received
0810 07:22:17.357558 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.357559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.357572 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.357644 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:17.358698 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.359288 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.360325 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.360554 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.360619 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.363335 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:17.363337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:17.363338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:17.363347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:17.365122 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.365721 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.365723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.365723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.365805 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.366156 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.366157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.366158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.366329 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:17.366415 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.366775 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.366886 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260810 07:22:17.367225 4602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.356603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:40209)
0810 07:22:17.356830 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.356905 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.357412 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.357729 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.357733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.358352 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:17.358580 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.358586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.359391 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.359392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.360232 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.360235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.360643 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.360644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.360810 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.361429 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:17.361438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:17.363178 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:17.365227 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:17.365236 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:17.365240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:17.365539 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:17.365924 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:17.365925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:17.365926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:17.366069 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:17.366421 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:17.366430 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:17.366602 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.366747 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.366948 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260810 07:22:17.367815 4600 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35471
I20260810 07:22:17.367954 4600 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:17.368167 4600 heartbeater.cc:511] Master 127.14.164.62:35471 requested a full tablet report, sending...
I20260810 07:22:17.368906 4298 ts_manager.cc:194] Registered new tserver with Master: 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753)
I20260810 07:22:17.369619 4298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:40209
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:22:17.397817 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35471
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:17.484495 4610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:17.484782 4610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:17.484824 4610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:17.486049 4610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:17.486127 4610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:17.486173 4610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:17.487696 4610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:17.487983 4610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:17.488245 4610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:17.493642 4617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.493664 4618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:17.493855 4610 server_base.cc:1034] running on GCE node
W20260810 07:22:17.493642 4623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:17.494144 4610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:17.494426 4610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:17.495604 4610 hybrid_clock.cc:648] HybridClock initialized: now 1786346537495584 us; error 76 us; skew 500 ppm
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:17.498555 4610 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:17.499111 4610 webserver.cc:460] Webserver started at http://127.14.164.2:36807/ using document root <none> and password file <none>
I20260810 07:22:17.499364 4610 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:17.499446 4610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:17.499607 4610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:17.500916 4610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "e4f125ea4ab64db6be83ed7566725bdb"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "73ba0a123cf2bd1348472b6607cf737f"
server_key_iv: "d25d20898cb86eca953e00454625f082"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.501410 4610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "e4f125ea4ab64db6be83ed7566725bdb"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "73ba0a123cf2bd1348472b6607cf737f"
server_key_iv: "d25d20898cb86eca953e00454625f082"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.502933 4610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:22:17.503818 4632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.504014 4610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:17.504129 4610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "e4f125ea4ab64db6be83ed7566725bdb"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "73ba0a123cf2bd1348472b6607cf737f"
server_key_iv: "d25d20898cb86eca953e00454625f082"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.504211 4610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:17.517906 4610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:17.520470 4610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:17.520679 4610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:17.521023 4610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:17.521386 4610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:17.521445 4610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.521486 4610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:17.521533 4610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.528712 4610 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:38185
I20260810 07:22:17.528793 4807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:38185 every 8 connection(s)
I20260810 07:22:17.529168 4610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:17.534814 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 4610
I20260810 07:22:17.534996 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:22:17.535239 14992 external_mini_cluster.cc:1428] Setting key 5990203816d89739626d014c2de55955
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:17.540361 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.529963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:50837 (local address 127.14.164.62:35471)
0810 07:22:17.530040 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.530042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.530279 (+ 237us) server_negotiation.cc:407] Connection header received
0810 07:22:17.530863 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.530865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.530878 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.530937 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:17.531968 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.532502 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.533558 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.533779 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.533857 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.536636 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:17.536637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:17.536638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:17.536647 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:17.538456 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.539111 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.539112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.539112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.539180 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.539518 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.539519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.539519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.539644 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:17.539707 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.540126 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.540233 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:22:17.540414 4810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.529946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:50837)
0810 07:22:17.530143 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.530183 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.530752 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.531018 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.531022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.531628 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:17.531844 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.531849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.532611 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.532612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.533459 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.533461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.533879 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.533880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.534018 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.534678 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:17.534691 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:17.536530 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:17.538553 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:17.538563 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:17.538566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:17.539001 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:17.539268 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:17.539269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:17.539269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:17.539441 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:17.539721 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:17.539728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:17.539933 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.540071 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.540218 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260810 07:22:17.541090 4808 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35471
I20260810 07:22:17.541222 4808 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:17.541456 4808 heartbeater.cc:511] Master 127.14.164.62:35471 requested a full tablet report, sending...
I20260810 07:22:17.541937 4298 ts_manager.cc:194] Registered new tserver with Master: e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
I20260810 07:22:17.542558 4298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:50837
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:22:17.569362 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35471
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:17.666715 4816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:17.666957 4816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:17.666985 4816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:17.668236 4816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:17.668275 4816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:17.668305 4816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:17.670004 4816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:17.670243 4816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:17.670452 4816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:17.675993 4829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.675983 4827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:17.675988 4835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:17.676339 4816 server_base.cc:1034] running on GCE node
I20260810 07:22:17.676607 4816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:17.676950 4816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:17.678122 4816 hybrid_clock.cc:648] HybridClock initialized: now 1786346537678091 us; error 43 us; skew 500 ppm
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:17.681090 4816 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:17.681681 4816 webserver.cc:460] Webserver started at http://127.14.164.3:40047/ using document root <none> and password file <none>
I20260810 07:22:17.681981 4816 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:17.682066 4816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:17.682235 4816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:17.683444 4816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "8356c4df9a624e73a0f929d9aa223d3a"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "be0a58e332249c7690c343544cc5f4cd"
server_key_iv: "3f0d8d3ce6a84d63d38eac3a61d00481"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.683862 4816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "8356c4df9a624e73a0f929d9aa223d3a"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "be0a58e332249c7690c343544cc5f4cd"
server_key_iv: "3f0d8d3ce6a84d63d38eac3a61d00481"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.685648 4816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 07:22:17.686691 4846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.686921 4816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:17.687036 4816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "8356c4df9a624e73a0f929d9aa223d3a"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "be0a58e332249c7690c343544cc5f4cd"
server_key_iv: "3f0d8d3ce6a84d63d38eac3a61d00481"
server_key_version: "encryptionkey@0"
I20260810 07:22:17.687124 4816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:17.698055 4816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:17.700294 4816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:17.700484 4816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:17.700850 4816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:17.701208 4816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:17.701270 4816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.701313 4816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:17.701342 4816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:17.707983 4816 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:34999
I20260810 07:22:17.708032 5023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:34999 every 8 connection(s)
I20260810 07:22:17.708386 4816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:22:17 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346537, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:17.716601 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 4816
I20260810 07:22:17.716710 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:22:17.716908 14992 external_mini_cluster.cc:1428] Setting key 942072c9180eb65cbae9697e66efdee7
I20260810 07:22:17.719040 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.709145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:57521 (local address 127.14.164.62:35471)
0810 07:22:17.709213 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.709239 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.709449 (+ 210us) server_negotiation.cc:407] Connection header received
0810 07:22:17.710568 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.710570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.710586 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.710638 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:17.711357 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.711919 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.713122 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.713381 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.713484 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.715798 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:17.715799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:17.715799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:17.715806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:17.717641 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.718135 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.718136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.718137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.718195 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.718421 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.718422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.718422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.718529 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:17.718580 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.718884 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.718974 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:22:17.719149 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.709102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:57521)
0810 07:22:17.709316 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.709356 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.710433 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.710728 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.710738 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.711024 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:17.711237 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.711241 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.712030 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.712031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.713021 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.713025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.713498 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.713499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.713638 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.714161 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:17.714170 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:17.715692 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:17.717720 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:17.717728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:17.717730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:17.718065 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:17.718258 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:17.718258 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:17.718259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:17.718365 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:17.718590 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:17.718596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:17.718721 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.718815 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.718952 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260810 07:22:17.719715 5024 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35471
I20260810 07:22:17.719830 5024 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:17.720077 5024 heartbeater.cc:511] Master 127.14.164.62:35471 requested a full tablet report, sending...
I20260810 07:22:17.720481 4298 ts_manager.cc:194] Registered new tserver with Master: 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999)
I20260810 07:22:17.720932 4298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:57521
I20260810 07:22:17.729120 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:17.737263 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.730678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.14.164.62:35471)
0810 07:22:17.730782 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.730784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.730875 (+ 91us) server_negotiation.cc:407] Connection header received
0810 07:22:17.730959 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.730960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.730971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.731035 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:17.731650 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.732154 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.732979 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.733182 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.733256 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.734492 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:17.734494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:17.734494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:17.734502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:17.735638 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.736215 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.736217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.736218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.736288 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:17.736525 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:17.736526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:17.736526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:17.736694 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:17.736877 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.737064 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.737155 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 07:22:17.739084 4298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:17.740813 4298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:17.747327 5047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.743098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:34999 (local address 127.0.0.1:53936)
0810 07:22:17.743277 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.743318 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.743388 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.743747 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.743748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.743751 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:17.743807 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.743809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.744930 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.744931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.746166 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.746169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.747066 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.747067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.747130 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.747136 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.747186 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.747229 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":1,"thread_start_us":75,"threads_started":1}
I20260810 07:22:17.747334 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.743229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.14.164.3:34999)
0810 07:22:17.743583 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.743587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.743596 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:22:17.743635 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.743637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.743650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.743894 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:17.743977 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.744780 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.746261 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.746974 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.747048 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.747103 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.747211 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.747256 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":87,"threads_started":1}
I20260810 07:22:17.747421 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.743276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.14.164.2:38185)
0810 07:22:17.743500 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.743506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.743517 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:17.743563 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.743565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.743582 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.743690 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:17.743914 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.744687 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.746021 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.746772 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.746845 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.746908 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.747281 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.747325 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":96,"threads_started":1}
I20260810 07:22:17.747452 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.743128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:38185 (local address 127.0.0.1:39550)
0810 07:22:17.743409 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.743419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.743474 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.743706 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.743710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.743713 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:17.743819 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.743821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.744840 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.744841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.745918 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.745930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.746960 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.746962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.747054 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.747070 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.747137 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.747197 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":4,"thread_start_us":94,"threads_started":1}
I20260810 07:22:17.748162 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.743338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34753 (local address 127.0.0.1:58282)
0810 07:22:17.743751 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.743761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.743831 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.745094 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.745095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.745098 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:17.745145 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.745146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.746362 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.746363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.747085 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.747086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.747901 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.747902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.747966 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.747970 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.748036 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.748082 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":63,"threads_started":1}
I20260810 07:22:17.748381 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.743564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58282 (local address 127.14.164.1:34753)
0810 07:22:17.744864 (+ 1300us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.744875 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.744885 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:17.744945 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.744947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.744963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.745071 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:17.745213 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.746253 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.747181 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.747797 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.747887 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.747942 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.748251 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.748310 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1260,"thread_start_us":89,"threads_started":1}
I20260810 07:22:17.748636 4931 tablet_service.cc:1467] Processing CreateTablet for tablet eac76a841f294df8a9335f50208289aa (DEFAULT_TABLE table=test-table [id=a8e89c8e4e234e71a847503febdd8e41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:17.748643 4707 tablet_service.cc:1467] Processing CreateTablet for tablet eac76a841f294df8a9335f50208289aa (DEFAULT_TABLE table=test-table [id=a8e89c8e4e234e71a847503febdd8e41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:17.749060 4707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac76a841f294df8a9335f50208289aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:17.749061 4931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac76a841f294df8a9335f50208289aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:17.751780 5057 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Bootstrap starting.
I20260810 07:22:17.751920 5058 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Bootstrap starting.
I20260810 07:22:17.752420 5057 tablet_bootstrap.cc:654] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:17.752395 4494 tablet_service.cc:1467] Processing CreateTablet for tablet eac76a841f294df8a9335f50208289aa (DEFAULT_TABLE table=test-table [id=a8e89c8e4e234e71a847503febdd8e41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:17.752651 5058 tablet_bootstrap.cc:654] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:17.752893 4494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac76a841f294df8a9335f50208289aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:17.753108 5057 log.cc:824] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:17.753290 5058 log.cc:824] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:17.753979 5057 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: No bootstrap required, opened a new log
I20260810 07:22:17.754061 5057 ts_tablet_manager.cc:1397] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:17.754068 5058 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: No bootstrap required, opened a new log
I20260810 07:22:17.754130 5058 ts_tablet_manager.cc:1397] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:17.755818 5058 raft_consensus.cc:348] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.755980 5058 raft_consensus.cc:374] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:17.756026 5058 raft_consensus.cc:729] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f125ea4ab64db6be83ed7566725bdb, State: Initialized, Role: FOLLOWER
I20260810 07:22:17.756137 5058 consensus_queue.cc:260] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.756201 5057 raft_consensus.cc:348] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.756381 5057 raft_consensus.cc:374] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:17.756443 5057 raft_consensus.cc:729] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8356c4df9a624e73a0f929d9aa223d3a, State: Initialized, Role: FOLLOWER
I20260810 07:22:17.756564 5057 consensus_queue.cc:260] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.756624 5058 ts_tablet_manager.cc:1428] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:22:17.756659 4808 heartbeater.cc:503] Master 127.14.164.62:35471 was elected leader, sending a full tablet report...
I20260810 07:22:17.756889 5057 ts_tablet_manager.cc:1428] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:22:17.757014 5024 heartbeater.cc:503] Master 127.14.164.62:35471 was elected leader, sending a full tablet report...
I20260810 07:22:17.757284 5064 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap starting.
I20260810 07:22:17.758024 5064 tablet_bootstrap.cc:654] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:17.758591 5064 log.cc:824] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:17.759375 5064 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: No bootstrap required, opened a new log
I20260810 07:22:17.759440 5064 ts_tablet_manager.cc:1397] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:17.760959 5064 raft_consensus.cc:348] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.761091 5064 raft_consensus.cc:374] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:17.761113 5064 raft_consensus.cc:729] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abbce2e828b46e9a78ec38bc532dc86, State: Initialized, Role: FOLLOWER
I20260810 07:22:17.761276 5064 consensus_queue.cc:260] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.761580 5064 ts_tablet_manager.cc:1428] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:17.761621 4600 heartbeater.cc:503] Master 127.14.164.62:35471 was elected leader, sending a full tablet report...
W20260810 07:22:17.779830 4809 tablet.cc:2369] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:17.847555 5067 raft_consensus.cc:482] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:17.847712 5067 raft_consensus.cc:504] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.848068 5067 leader_election.cc:288] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185), 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753)
I20260810 07:22:17.851536 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.848467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:46499 (local address 127.14.164.2:38185)
0810 07:22:17.848554 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.848557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.848571 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:22:17.848607 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.848608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.848622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.848711 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:17.848980 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.849568 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.850699 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.851115 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.851185 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.851193 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.851408 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.851456 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:22:17.851573 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.848585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:47469 (local address 127.14.164.1:34753)
0810 07:22:17.848676 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.848679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.848873 (+ 194us) server_negotiation.cc:407] Connection header received
0810 07:22:17.848907 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.848908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.848924 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.848987 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:17.849176 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.849756 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.850781 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.851143 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.851256 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.851265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.851460 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.851508 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:22:17.851568 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.848342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:38185 (local address 127.14.164.3:46499)
0810 07:22:17.848415 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.848429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.848485 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.848738 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.848740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.848742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:17.848858 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.848860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.849663 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.849664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.850588 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.850591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.851215 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.851216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.851319 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.851323 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.851379 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.851439 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":22}
I20260810 07:22:17.851584 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.848489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34753 (local address 127.14.164.3:47469)
0810 07:22:17.848768 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.848780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:17.848857 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:17.849015 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:17.849016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:17.849018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:17.849100 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.849102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.849859 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.849860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.850622 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:17.850624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.851270 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.851271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:17.851350 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.851354 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:17.851432 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:22:17.851470 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
I20260810 07:22:17.852039 4532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac76a841f294df8a9335f50208289aa" candidate_uuid: "8356c4df9a624e73a0f929d9aa223d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abbce2e828b46e9a78ec38bc532dc86" is_pre_election: true
I20260810 07:22:17.852020 4737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac76a841f294df8a9335f50208289aa" candidate_uuid: "8356c4df9a624e73a0f929d9aa223d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f125ea4ab64db6be83ed7566725bdb" is_pre_election: true
I20260810 07:22:17.852217 4532 raft_consensus.cc:2454] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8356c4df9a624e73a0f929d9aa223d3a in term 0.
I20260810 07:22:17.852255 4737 raft_consensus.cc:2454] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8356c4df9a624e73a0f929d9aa223d3a in term 0.
I20260810 07:22:17.852530 4847 leader_election.cc:302] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb; no voters:
I20260810 07:22:17.852669 5067 raft_consensus.cc:2781] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:17.852799 5067 raft_consensus.cc:482] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:17.852840 5067 raft_consensus.cc:3037] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:17.853741 5067 raft_consensus.cc:504] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:17.853893 5067 leader_election.cc:288] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [CANDIDATE]: Term 1 election: Requested vote from peers e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185), 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753)
I20260810 07:22:17.854128 4737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac76a841f294df8a9335f50208289aa" candidate_uuid: "8356c4df9a624e73a0f929d9aa223d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f125ea4ab64db6be83ed7566725bdb"
I20260810 07:22:17.854169 4532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac76a841f294df8a9335f50208289aa" candidate_uuid: "8356c4df9a624e73a0f929d9aa223d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abbce2e828b46e9a78ec38bc532dc86"
I20260810 07:22:17.854209 4737 raft_consensus.cc:3037] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:17.854250 4532 raft_consensus.cc:3037] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:17.855103 4532 raft_consensus.cc:2454] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8356c4df9a624e73a0f929d9aa223d3a in term 1.
I20260810 07:22:17.855219 4737 raft_consensus.cc:2454] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8356c4df9a624e73a0f929d9aa223d3a in term 1.
I20260810 07:22:17.855319 4847 leader_election.cc:302] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86, 8356c4df9a624e73a0f929d9aa223d3a; no voters:
I20260810 07:22:17.855432 5067 raft_consensus.cc:2781] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:17.855656 5067 raft_consensus.cc:686] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 1 LEADER]: Becoming Leader. State: Replica: 8356c4df9a624e73a0f929d9aa223d3a, State: Running, Role: LEADER
I20260810 07:22:17.855772 5067 consensus_queue.cc:237] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
W20260810 07:22:17.856564 4601 tablet.cc:2369] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:17.856967 4298 catalog_manager.cc:5495] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3). New cstate: current_term: 1 leader_uuid: "8356c4df9a624e73a0f929d9aa223d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260810 07:22:17.864138 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.861629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.14.164.3:34999)
0810 07:22:17.861695 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:17.861697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:17.861705 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:22:17.861765 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:17.861765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:17.861782 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:17.861844 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:17.862040 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.862504 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.863258 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:17.863453 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:17.863523 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:17.863705 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:17.863804 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:17.863882 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:17.863961 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 07:22:17.863989 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:22:17.866135 4532 raft_consensus.cc:1261] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Refusing update from remote peer 8356c4df9a624e73a0f929d9aa223d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:17.866135 4737 raft_consensus.cc:1261] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Refusing update from remote peer 8356c4df9a624e73a0f929d9aa223d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:17.866534 5071 consensus_queue.cc:1035] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:17.866683 5067 consensus_queue.cc:1035] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:17.869674 5075 mvcc.cc:205] Tried to move back new op lower bound from 7316875419096567808 to 7316875419058167808. Current Snapshot: MvccSnapshot[applied={T|T < 7316875419096567808}]
I20260810 07:22:17.871482 4295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:17.871572 4295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:17.872349 4295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Sending DeleteTablet for 3 replicas of tablet eac76a841f294df8a9335f50208289aa
I20260810 07:22:17.872699 4931 tablet_service.cc:1514] Processing DeleteTablet for tablet eac76a841f294df8a9335f50208289aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:53936
I20260810 07:22:17.872753 4494 tablet_service.cc:1514] Processing DeleteTablet for tablet eac76a841f294df8a9335f50208289aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:58282
I20260810 07:22:17.872978 5090 tablet_replica.cc:330] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: stopping tablet replica
I20260810 07:22:17.873032 4707 tablet_service.cc:1514] Processing DeleteTablet for tablet eac76a841f294df8a9335f50208289aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:39550
I20260810 07:22:17.873091 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 4401
I20260810 07:22:17.873133 5090 raft_consensus.cc:2229] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:22:17.893169 5090 raft_consensus.cc:2258] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:17.893895 5090 ts_tablet_manager.cc:1905] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:17.895823 5090 ts_tablet_manager.cc:1918] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:17.895918 5090 log.cc:1196] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/eac76a841f294df8a9335f50208289aa
I20260810 07:22:17.896214 5090 ts_tablet_manager.cc:1939] T eac76a841f294df8a9335f50208289aa P 8356c4df9a624e73a0f929d9aa223d3a: Deleting consensus metadata
I20260810 07:22:17.896816 4278 catalog_manager.cc:4841] TS 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): tablet eac76a841f294df8a9335f50208289aa (table test-table [id=a8e89c8e4e234e71a847503febdd8e41]) successfully deleted
W20260810 07:22:17.898125 4277 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:34753 (error 108)
I20260810 07:22:17.898478 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 4610
I20260810 07:22:17.898538 5091 tablet_replica.cc:330] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: stopping tablet replica
I20260810 07:22:17.898685 5091 raft_consensus.cc:2229] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:17.898731 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.898582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34753 (local address 127.0.0.1:58296)
0810 07:22:17.898640 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.898679 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:22:17.898845 4277 catalog_manager.cc:4584] TS 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eac76a841f294df8a9335f50208289aa: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34753: connect: Connection refused (error 111)
I20260810 07:22:17.898864 5091 raft_consensus.cc:2258] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:17.914036 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 4816
I20260810 07:22:17.914151 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:17.913966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:38185 (local address 127.0.0.1:39560)
0810 07:22:17.914053 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:17.914083 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.2:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 07:22:17.924913 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 4253
I20260810 07:22:17.936434 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35471
--webserver_interface=127.14.164.62
--webserver_port=38499
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:18.024474 5093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:18.024708 5093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:18.024766 5093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:18.026006 5093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:18.026051 5093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:18.026067 5093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:18.026080 5093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:18.026093 5093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:18.027544 5093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=38499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:18.027757 5093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:18.027932 5093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:18.038192 5107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.038175 5101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.038175 5100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:18.038463 5093 server_base.cc:1034] running on GCE node
I20260810 07:22:18.038910 5093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:18.039278 5093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:18.040444 5093 hybrid_clock.cc:648] HybridClock initialized: now 1786346538040422 us; error 38 us; skew 500 ppm
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:18.043267 5093 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:18.043787 5093 webserver.cc:460] Webserver started at http://127.14.164.62:38499/ using document root <none> and password file <none>
I20260810 07:22:18.044054 5093 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:18.044137 5093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:18.045962 5093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:18.046900 5119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.047115 5093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:18.047245 5093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "76cd9be632a4280f974d26adad6c9f2e"
server_key_iv: "520eaa813d9de19cadc132f321db4b2d"
server_key_version: "encryptionkey@0"
I20260810 07:22:18.047783 5093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:18.070204 5093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:18.072324 5093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:18.072566 5093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:18.077324 5093 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:35471
I20260810 07:22:18.077381 5191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:35471 every 8 connection(s)
I20260810 07:22:18.077862 5093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:18.079672 5192 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:18.080408 5192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap starting.
I20260810 07:22:18.082867 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 5093
I20260810 07:22:18.083187 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:34753
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:35471
--webserver_port=42699
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:18.090365 5192 log.cc:824] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:18.092449 5192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:18.092868 5192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap complete.
I20260810 07:22:18.095347 5192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:18.095599 5192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0bd29f032548be8dc8ac7eb6ff5f33, State: Initialized, Role: FOLLOWER
I20260810 07:22:18.095774 5192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:18.095849 5192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:18.095921 5192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:18.095970 5192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:18.097102 5192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:18.097229 5192 leader_election.cc:302] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a0bd29f032548be8dc8ac7eb6ff5f33; no voters:
I20260810 07:22:18.097470 5192 leader_election.cc:288] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:22:18.097517 5196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:22:18.097689 5196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 LEADER]: Becoming Leader. State: Replica: 4a0bd29f032548be8dc8ac7eb6ff5f33, State: Running, Role: LEADER
I20260810 07:22:18.097759 5192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:18.097896 5196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:18.098282 5200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } } }
I20260810 07:22:18.098426 5200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:18.098673 5212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:18.098284 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a0bd29f032548be8dc8ac7eb6ff5f33. Latest consensus state: current_term: 2 leader_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } } }
I20260810 07:22:18.098918 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:18.099731 5212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:18.099897 5212 catalog_manager.cc:1259] Loaded cluster ID: b8834e0ed5714912bc5b3a15f49d94d4
I20260810 07:22:18.100085 5212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:18.100390 5212 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 07:22:18.105477 5212 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 07:22:18.108016 5212 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:18.108228 5212 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Loaded TSK: 0
I20260810 07:22:18.108445 5212 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:22:18.175459 5194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:18.175770 5194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:18.175825 5194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:18.178047 5194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:18.178123 5194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:18.178176 5194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:18.180881 5194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:34753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=42699
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:18.182693 5194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:18.182976 5194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:18.189435 5242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.189473 5238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.189435 5244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:18.189954 5194 server_base.cc:1034] running on GCE node
I20260810 07:22:18.190201 5194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:18.190490 5194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:18.191658 5194 hybrid_clock.cc:648] HybridClock initialized: now 1786346538191645 us; error 28 us; skew 500 ppm
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:18.194278 5194 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:18.194818 5194 webserver.cc:460] Webserver started at http://127.14.164.1:42699/ using document root <none> and password file <none>
I20260810 07:22:18.195087 5194 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:18.195144 5194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:18.196570 5194 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:18.197340 5253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.197556 5194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.197659 5194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "6abbce2e828b46e9a78ec38bc532dc86"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "37021986e815b7a35343af958facfaef"
server_key_iv: "71bea22e951f0dbd62909bff02acbe7b"
server_key_version: "encryptionkey@0"
I20260810 07:22:18.198071 5194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:18.208350 5194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:18.210685 5194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:18.210863 5194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:18.211143 5194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:18.211665 5264 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:18.213270 5194 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:18.213322 5194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:22:18.213358 5194 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:18.214140 5194 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:18.214188 5194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.214237 5264 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap starting.
I20260810 07:22:18.222443 5194 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:34753
I20260810 07:22:18.222658 5439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:34753 every 8 connection(s)
I20260810 07:22:18.223037 5194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:18.226999 5264 log.cc:824] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:18.229404 5264 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:18.229717 5264 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap complete.
I20260810 07:22:18.230902 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 5194
I20260810 07:22:18.231020 5264 ts_tablet_manager.cc:1397] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260810 07:22:18.231201 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:38185
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:35471
--webserver_port=36807
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:18.232165 5264 raft_consensus.cc:348] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.232447 5264 raft_consensus.cc:729] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abbce2e828b46e9a78ec38bc532dc86, State: Initialized, Role: FOLLOWER
I20260810 07:22:18.232596 5264 consensus_queue.cc:260] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.232923 5264 ts_tablet_manager.cc:1428] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:18.239625 5447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.223777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:54003)
0810 07:22:18.224011 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.224061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.224655 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.225589 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.225593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.225998 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:18.226263 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.226270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.227898 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.227900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.229224 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.229228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.230077 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.230078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.230241 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.233235 (+ 2994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:18.233244 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:18.234807 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:18.237434 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.237443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.237446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.237811 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.238145 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.238146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.238146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.238291 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.238847 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:18.238854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:18.239035 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.239155 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.239356 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":137,"threads_started":1}
I20260810 07:22:18.241220 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.223925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:54003 (local address 127.14.164.62:35471)
0810 07:22:18.225039 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.225050 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.225107 (+ 57us) server_negotiation.cc:407] Connection header received
0810 07:22:18.225273 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.225283 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.225360 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.225535 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:18.226410 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.227768 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.229324 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.229962 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.230072 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.234927 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:18.234932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:18.234933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:18.234946 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:18.237325 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.237918 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.237922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.237922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.238062 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.238374 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.238375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.238375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.238708 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:18.238814 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.240906 (+ 2092us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.241047 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":85,"threads_started":1}
I20260810 07:22:18.241849 5440 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35471
I20260810 07:22:18.241998 5440 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:18.242188 5440 heartbeater.cc:511] Master 127.14.164.62:35471 requested a full tablet report, sending...
I20260810 07:22:18.243047 5143 ts_manager.cc:194] Registered new tserver with Master: 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753)
I20260810 07:22:18.244357 5143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:54003
I20260810 07:22:18.245342 5440 heartbeater.cc:503] Master 127.14.164.62:35471 was elected leader, sending a full tablet report...
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.1@KRBTEST.COM
I20260810 07:22:18.252074 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.243754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34753 (local address 127.0.0.1:58300)
0810 07:22:18.243945 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.243980 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.244055 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.244218 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.244221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.244559 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:18.244619 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.244622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.245850 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.245851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.246440 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.246442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.246951 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.246952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.247039 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.247560 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:18.247564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:18.248931 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:18.250774 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.250782 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.250785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.251065 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.251285 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.251286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.251286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.251360 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.251675 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:18.251683 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:18.251738 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.251839 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.251971 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":134,"threads_started":1}
I20260810 07:22:18.252074 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.243815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58300 (local address 127.14.164.1:34753)
0810 07:22:18.244091 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.244096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.244106 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:18.244140 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.244142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.244155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.244218 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:18.244777 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.245783 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.246532 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.246845 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.246936 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.249023 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:18.249031 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:18.249032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:18.249039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:18.250629 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.251138 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.251141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.251141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.251211 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.251428 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.251429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.251429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.251592 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:18.251684 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.251879 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.251981 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":84,"threads_started":1}
I20260810 07:22:18.252501 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet eac76a841f294df8a9335f50208289aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:58300
I20260810 07:22:18.253120 5456 tablet_replica.cc:330] stopping tablet replica
I20260810 07:22:18.253237 5456 raft_consensus.cc:2229] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:18.253310 5456 raft_consensus.cc:2258] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:18.253634 5456 ts_tablet_manager.cc:1905] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:18.255925 5456 ts_tablet_manager.cc:1918] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:18.256008 5456 log.cc:1196] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/eac76a841f294df8a9335f50208289aa
I20260810 07:22:18.256244 5456 ts_tablet_manager.cc:1939] T eac76a841f294df8a9335f50208289aa P 6abbce2e828b46e9a78ec38bc532dc86: Deleting consensus metadata
I20260810 07:22:18.256853 5120 catalog_manager.cc:4841] TS 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753): tablet eac76a841f294df8a9335f50208289aa (table test-table [id=a8e89c8e4e234e71a847503febdd8e41]) successfully deleted
W20260810 07:22:18.322979 5453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:18.323191 5453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:18.323215 5453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:18.324460 5453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:18.324505 5453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:18.324532 5453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:18.325964 5453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:38185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=36807
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:18.326153 5453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:18.326341 5453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:18.331925 5463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.331918 5461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:18.332011 5453 server_base.cc:1034] running on GCE node
W20260810 07:22:18.331930 5465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:18.332401 5453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:18.332700 5453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:18.333910 5453 hybrid_clock.cc:648] HybridClock initialized: now 1786346538333868 us; error 55 us; skew 500 ppm
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:18.336679 5453 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:18.337203 5453 webserver.cc:460] Webserver started at http://127.14.164.2:36807/ using document root <none> and password file <none>
I20260810 07:22:18.337457 5453 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:18.337522 5453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:18.339036 5453 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:22:18.339812 5476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.340015 5453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.340116 5453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "e4f125ea4ab64db6be83ed7566725bdb"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "73ba0a123cf2bd1348472b6607cf737f"
server_key_iv: "d25d20898cb86eca953e00454625f082"
server_key_version: "encryptionkey@0"
I20260810 07:22:18.340502 5453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:18.373772 5453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:18.376410 5453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:18.376538 5453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:18.376891 5453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:18.377409 5486 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:18.379133 5453 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:18.379225 5453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:22:18.379274 5453 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:18.380092 5453 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:18.380179 5453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.380264 5486 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Bootstrap starting.
I20260810 07:22:18.388161 5453 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:38185
I20260810 07:22:18.388299 5652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:38185 every 8 connection(s)
I20260810 07:22:18.388671 5453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:18.390493 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 5453
I20260810 07:22:18.390821 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:34999
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:35471
--webserver_port=40047
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:18.392096 5486 log.cc:824] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:18.394443 5486 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:18.394841 5486 tablet_bootstrap.cc:492] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Bootstrap complete.
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:18.396258 5486 ts_tablet_manager.cc:1397] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260810 07:22:18.397627 5486 raft_consensus.cc:348] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.397955 5486 raft_consensus.cc:729] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f125ea4ab64db6be83ed7566725bdb, State: Initialized, Role: FOLLOWER
I20260810 07:22:18.398155 5486 consensus_queue.cc:260] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.398432 5486 ts_tablet_manager.cc:1428] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 07:22:18.399967 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.389651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:47559 (local address 127.14.164.62:35471)
0810 07:22:18.389705 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.389708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.389956 (+ 248us) server_negotiation.cc:407] Connection header received
0810 07:22:18.390576 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.390578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.390597 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.390661 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:18.391563 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.392143 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.393548 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.393793 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.393892 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.396712 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:18.396714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:18.396715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:18.396754 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:18.398460 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.398951 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.398953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.398954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.399022 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.399288 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.399289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.399289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.399417 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:18.399487 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.399775 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.399872 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:22:18.400153 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.389566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:47559)
0810 07:22:18.389796 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.389851 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.390452 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.390779 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.390783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.391221 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:18.391448 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.391454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.392321 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.392322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.393434 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.393438 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.393928 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.393930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.394077 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.394685 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:18.394695 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:18.396596 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:18.398542 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.398546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.398548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.398852 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.399092 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.399093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.399093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.399211 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.399492 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:18.399499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:18.399641 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.399754 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.399931 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260810 07:22:18.400616 5653 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35471
I20260810 07:22:18.400759 5653 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:18.400974 5653 heartbeater.cc:511] Master 127.14.164.62:35471 requested a full tablet report, sending...
I20260810 07:22:18.401688 5143 ts_manager.cc:194] Registered new tserver with Master: e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
I20260810 07:22:18.402293 5143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:47559
I20260810 07:22:18.403141 5653 heartbeater.cc:503] Master 127.14.164.62:35471 was elected leader, sending a full tablet report...
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.2@KRBTEST.COM
I20260810 07:22:18.410591 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.402060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:38185 (local address 127.0.0.1:39574)
0810 07:22:18.402123 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.402136 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.402180 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.402675 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.402676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.402847 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:18.402894 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.402896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.404242 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.404243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.404817 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.404819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.405527 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.405528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.405588 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.406079 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:18.406084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:18.407574 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:18.409539 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.409540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.409540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.409708 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.409919 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.409920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.409920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.409973 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.410306 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:18.410307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:18.410378 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.410444 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.410524 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 07:22:18.410792 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.402285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.14.164.2:38185)
0810 07:22:18.402475 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.402481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.402493 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:22:18.402531 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.402533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.402552 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.402667 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:18.402986 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.404115 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.404904 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.405421 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.405506 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.407669 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:18.407679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:18.407679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:18.407687 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:18.409447 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.409786 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.409789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.409790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.409855 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.410032 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.410033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.410033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.410205 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:18.410316 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.410502 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.410603 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260810 07:22:18.411130 5553 tablet_service.cc:1514] Processing DeleteTablet for tablet eac76a841f294df8a9335f50208289aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:17 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:39574
I20260810 07:22:18.411651 5666 tablet_replica.cc:330] stopping tablet replica
I20260810 07:22:18.411773 5666 raft_consensus.cc:2229] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:18.411870 5666 raft_consensus.cc:2258] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:18.412192 5666 ts_tablet_manager.cc:1905] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:18.414407 5666 ts_tablet_manager.cc:1918] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:18.414497 5666 log.cc:1196] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/eac76a841f294df8a9335f50208289aa
I20260810 07:22:18.414758 5666 ts_tablet_manager.cc:1939] T eac76a841f294df8a9335f50208289aa P e4f125ea4ab64db6be83ed7566725bdb: Deleting consensus metadata
I20260810 07:22:18.415295 5120 catalog_manager.cc:4841] TS e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): tablet eac76a841f294df8a9335f50208289aa (table test-table [id=a8e89c8e4e234e71a847503febdd8e41]) successfully deleted
W20260810 07:22:18.484942 5663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:18.485177 5663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:18.485203 5663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:18.486435 5663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:18.486481 5663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:18.486509 5663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:18.488041 5663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:34999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=40047
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:18.488322 5663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:18.488600 5663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:18.494345 5673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.494379 5678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:18.494345 5672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:18.494829 5663 server_base.cc:1034] running on GCE node
I20260810 07:22:18.495011 5663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:18.495260 5663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:18.496413 5663 hybrid_clock.cc:648] HybridClock initialized: now 1786346538496399 us; error 31 us; skew 500 ppm
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:18.499431 5663 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:18.499909 5663 webserver.cc:460] Webserver started at http://127.14.164.3:40047/ using document root <none> and password file <none>
I20260810 07:22:18.500124 5663 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:18.500177 5663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:18.501688 5663 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.502362 5687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.502542 5663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.502655 5663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "8356c4df9a624e73a0f929d9aa223d3a"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "be0a58e332249c7690c343544cc5f4cd"
server_key_iv: "3f0d8d3ce6a84d63d38eac3a61d00481"
server_key_version: "encryptionkey@0"
I20260810 07:22:18.503029 5663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:18.550508 5663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:18.553421 5663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:18.553619 5663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:18.553900 5663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:18.554257 5663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:18.554317 5663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.554356 5663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:18.554386 5663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:18.560772 5663 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:34999
I20260810 07:22:18.560829 5871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:34999 every 8 connection(s)
I20260810 07:22:18.561236 5663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:18.563023 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 5663
Aug 10 07:22:18 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346538, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:18.572201 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.563512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.14.164.62:35471)
0810 07:22:18.563634 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.563636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.563780 (+ 144us) server_negotiation.cc:407] Connection header received
0810 07:22:18.563827 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.563828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.563837 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.563901 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:18.564651 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.565558 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.566853 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.567116 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.567195 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.568967 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:18.568968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:18.568969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:18.568976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:18.570786 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.571295 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.571297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.571297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.571355 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.571605 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.571606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.571606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.571716 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:18.571831 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.572004 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.572099 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260810 07:22:18.573647 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.562109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:54791 (local address 127.14.164.62:35471)
0810 07:22:18.562201 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.562204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.562380 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:22:18.563031 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.563033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.563047 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.563125 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:18.564651 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.565553 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.567052 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.567329 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.567379 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.569936 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:18.569938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:18.569938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:18.569946 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:18.571854 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.572488 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.572490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.572490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.572562 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:18.572929 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:18.572930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:18.572931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:18.573060 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:18.573142 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.573461 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.573535 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":53}
I20260810 07:22:18.573766 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.562051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:54791)
0810 07:22:18.562243 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.562289 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.562824 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.563256 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.563260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.564187 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:18.564532 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.564539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.565687 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.565688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.566966 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.566970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.567422 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.567424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.567586 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.568071 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:18.568082 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:18.569833 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:18.571981 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.571990 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.571993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.572364 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.572638 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:18.572639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:18.572640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:18.572850 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:18.573133 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:18.573139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:18.573320 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.573420 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.573555 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260810 07:22:18.574261 5873 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35471
I20260810 07:22:18.574380 5873 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:18.574577 5873 heartbeater.cc:511] Master 127.14.164.62:35471 requested a full tablet report, sending...
I20260810 07:22:18.575052 5143 ts_manager.cc:194] Registered new tserver with Master: 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999)
I20260810 07:22:18.575392 5143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:54791
I20260810 07:22:18.584133 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:18.587141 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.584617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.14.164.62:35471)
0810 07:22:18.584674 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.584676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.584833 (+ 157us) server_negotiation.cc:407] Connection header received
0810 07:22:18.584903 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.584904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.584941 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.585001 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:18.585101 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.585620 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.586452 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.586636 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.586696 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.586855 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:18.586965 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:18.587030 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.587066 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.587083 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:22:18.587455 5143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:18.589229 5143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:18.591727 5337 tablet_service.cc:1467] Processing CreateTablet for tablet a8c25fa8033243aeb74eac530f8df235 (DEFAULT_TABLE table=test-table [id=03c12fe7369444d580c193c8009c23fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:18.591977 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c25fa8033243aeb74eac530f8df235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:18.592042 5553 tablet_service.cc:1467] Processing CreateTablet for tablet a8c25fa8033243aeb74eac530f8df235 (DEFAULT_TABLE table=test-table [id=03c12fe7369444d580c193c8009c23fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:18.592199 5553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c25fa8033243aeb74eac530f8df235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:18.593739 5264 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap starting.
I20260810 07:22:18.593811 5486 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Bootstrap starting.
I20260810 07:22:18.594110 5486 tablet_bootstrap.cc:654] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:18.594111 5264 tablet_bootstrap.cc:654] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:18.594621 5264 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: No bootstrap required, opened a new log
I20260810 07:22:18.594667 5264 ts_tablet_manager.cc:1397] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.594696 5486 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: No bootstrap required, opened a new log
I20260810 07:22:18.594732 5486 ts_tablet_manager.cc:1397] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.595024 5486 raft_consensus.cc:348] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.595032 5264 raft_consensus.cc:348] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.595085 5486 raft_consensus.cc:374] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:18.595115 5486 raft_consensus.cc:729] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f125ea4ab64db6be83ed7566725bdb, State: Initialized, Role: FOLLOWER
I20260810 07:22:18.595111 5264 raft_consensus.cc:374] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:18.595157 5264 raft_consensus.cc:729] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abbce2e828b46e9a78ec38bc532dc86, State: Initialized, Role: FOLLOWER
I20260810 07:22:18.595184 5486 consensus_queue.cc:260] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.595208 5264 consensus_queue.cc:260] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.595309 5264 ts_tablet_manager.cc:1428] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.595315 5486 ts_tablet_manager.cc:1428] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:18.595907 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.591498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:34999 (local address 127.0.0.1:53946)
0810 07:22:18.591554 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.591564 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.591603 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.592198 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.592200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.592202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:18.592248 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.592249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.593332 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.593333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.594334 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.594338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.595632 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.595633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.595704 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.595730 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.595782 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.595829 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:22:18.596185 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.591700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.14.164.3:34999)
0810 07:22:18.591967 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.591972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.591982 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:18.592029 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.592047 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.592066 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.592187 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:18.592313 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.593223 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.594455 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.595255 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.595332 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.595404 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.596065 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.596113 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":130,"threads_started":1}
I20260810 07:22:18.596997 5758 tablet_service.cc:1467] Processing CreateTablet for tablet a8c25fa8033243aeb74eac530f8df235 (DEFAULT_TABLE table=test-table [id=03c12fe7369444d580c193c8009c23fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:18.597345 5758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c25fa8033243aeb74eac530f8df235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:18.600634 5887 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Bootstrap starting.
I20260810 07:22:18.601305 5887 tablet_bootstrap.cc:654] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:18.601801 5887 log.cc:824] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:18.602537 5887 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: No bootstrap required, opened a new log
I20260810 07:22:18.602628 5887 ts_tablet_manager.cc:1397] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:18.604035 5887 raft_consensus.cc:348] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.604202 5887 raft_consensus.cc:374] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:18.604254 5887 raft_consensus.cc:729] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8356c4df9a624e73a0f929d9aa223d3a, State: Initialized, Role: FOLLOWER
I20260810 07:22:18.604400 5887 consensus_queue.cc:260] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.604701 5887 ts_tablet_manager.cc:1428] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:18.604749 5873 heartbeater.cc:503] Master 127.14.164.62:35471 was elected leader, sending a full tablet report...
W20260810 07:22:18.639359 5655 tablet.cc:2369] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:22:18.725384 5441 tablet.cc:2369] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:22:18.811937 5875 tablet.cc:2369] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:18.869952 5452 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:18.870131 5452 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.870565 5452 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
I20260810 07:22:18.874152 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.870903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:56551 (local address 127.14.164.3:34999)
0810 07:22:18.871009 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.871012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.871056 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:22:18.871183 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.871184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.871195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.871280 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:18.871617 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.872357 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.873371 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.873750 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.873811 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.873845 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.874021 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.874072 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 07:22:18.874158 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.870943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37093 (local address 127.14.164.2:38185)
0810 07:22:18.871057 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.871060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.871070 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:18.871197 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.871198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.871225 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.871311 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:18.871597 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.872162 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.873302 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.873758 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.873855 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.873958 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.874033 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.874077 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:22:18.874193 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.870817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:34999 (local address 127.14.164.1:56551)
0810 07:22:18.870980 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.870994 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.871087 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.871397 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.871398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.871402 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:18.871487 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.871488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.872470 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.872471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.873262 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.873265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.873845 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.873846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.873904 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.873935 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.873996 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.874064 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260810 07:22:18.874210 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.870834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:38185 (local address 127.14.164.1:37093)
0810 07:22:18.871004 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.871016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:18.871087 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:18.871397 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:18.871398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:18.871403 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:18.871478 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.871481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.872280 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.872281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.873204 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:18.873207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.873886 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.873887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:18.873950 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.873954 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:18.874028 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:22:18.874102 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":24,"thread_start_us":67,"threads_started":1}
I20260810 07:22:18.874573 5792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c25fa8033243aeb74eac530f8df235" candidate_uuid: "6abbce2e828b46e9a78ec38bc532dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8356c4df9a624e73a0f929d9aa223d3a" is_pre_election: true
I20260810 07:22:18.874720 5585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c25fa8033243aeb74eac530f8df235" candidate_uuid: "6abbce2e828b46e9a78ec38bc532dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f125ea4ab64db6be83ed7566725bdb" is_pre_election: true
I20260810 07:22:18.874804 5792 raft_consensus.cc:2454] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6abbce2e828b46e9a78ec38bc532dc86 in term 0.
I20260810 07:22:18.874949 5585 raft_consensus.cc:2454] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6abbce2e828b46e9a78ec38bc532dc86 in term 0.
I20260810 07:22:18.875057 5259 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86, 8356c4df9a624e73a0f929d9aa223d3a; no voters:
I20260810 07:22:18.875196 5452 raft_consensus.cc:2781] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:18.875273 5452 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:18.875293 5452 raft_consensus.cc:3037] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:18.876022 5452 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.876176 5452 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 1 election: Requested vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
I20260810 07:22:18.876338 5585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c25fa8033243aeb74eac530f8df235" candidate_uuid: "6abbce2e828b46e9a78ec38bc532dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f125ea4ab64db6be83ed7566725bdb"
I20260810 07:22:18.876441 5792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c25fa8033243aeb74eac530f8df235" candidate_uuid: "6abbce2e828b46e9a78ec38bc532dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8356c4df9a624e73a0f929d9aa223d3a"
I20260810 07:22:18.876469 5585 raft_consensus.cc:3037] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:18.876513 5792 raft_consensus.cc:3037] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:18.877480 5585 raft_consensus.cc:2454] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6abbce2e828b46e9a78ec38bc532dc86 in term 1.
I20260810 07:22:18.877480 5792 raft_consensus.cc:2454] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6abbce2e828b46e9a78ec38bc532dc86 in term 1.
I20260810 07:22:18.877691 5258 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86, e4f125ea4ab64db6be83ed7566725bdb; no voters:
I20260810 07:22:18.877869 5452 raft_consensus.cc:2781] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:18.878010 5452 raft_consensus.cc:686] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 LEADER]: Becoming Leader. State: Replica: 6abbce2e828b46e9a78ec38bc532dc86, State: Running, Role: LEADER
I20260810 07:22:18.878110 5452 consensus_queue.cc:237] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:18.878865 5143 catalog_manager.cc:5495] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "6abbce2e828b46e9a78ec38bc532dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260810 07:22:18.954457 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.951733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.14.164.1:34753)
0810 07:22:18.951898 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:18.951900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:18.951906 (+ 6us) server_negotiation.cc:407] Connection header received
0810 07:22:18.951978 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:18.951979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:18.952028 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:18.952139 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:18.952285 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.952825 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.953594 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:18.953810 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:18.953892 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:18.954087 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:18.954197 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:18.954285 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:18.954328 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:22:18.954350 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260810 07:22:18.956518 5585 raft_consensus.cc:1261] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Refusing update from remote peer 6abbce2e828b46e9a78ec38bc532dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:18.956600 5792 raft_consensus.cc:1261] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Refusing update from remote peer 6abbce2e828b46e9a78ec38bc532dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:18.956960 5452 consensus_queue.cc:1035] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:18.957129 5898 consensus_queue.cc:1035] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:18.962141 5137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:18.962275 5137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:18.963120 5137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Sending DeleteTablet for 3 replicas of tablet a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:18.963498 5758 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c25fa8033243aeb74eac530f8df235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:18 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:53946
I20260810 07:22:18.963536 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 5194
I20260810 07:22:18.963631 5553 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c25fa8033243aeb74eac530f8df235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:18 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:39574
I20260810 07:22:18.971674 5914 tablet_replica.cc:330] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: stopping tablet replica
I20260810 07:22:18.971654 5915 tablet_replica.cc:330] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: stopping tablet replica
I20260810 07:22:18.971763 5915 raft_consensus.cc:2229] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:18.971832 5915 raft_consensus.cc:2258] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:18.971853 5914 raft_consensus.cc:2229] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:18.972007 5914 raft_consensus.cc:2258] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:18.972383 5915 ts_tablet_manager.cc:1905] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:18.972615 5914 ts_tablet_manager.cc:1905] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:18.973973 5915 ts_tablet_manager.cc:1918] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:18.974032 5915 log.cc:1196] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:18.974287 5915 ts_tablet_manager.cc:1939] T a8c25fa8033243aeb74eac530f8df235 P e4f125ea4ab64db6be83ed7566725bdb: Deleting consensus metadata
I20260810 07:22:18.974695 5120 catalog_manager.cc:4841] TS e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): tablet a8c25fa8033243aeb74eac530f8df235 (table test-table [id=03c12fe7369444d580c193c8009c23fe]) successfully deleted
I20260810 07:22:18.975067 5914 ts_tablet_manager.cc:1918] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:18.975183 5914 log.cc:1196] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:18.975575 5914 ts_tablet_manager.cc:1939] T a8c25fa8033243aeb74eac530f8df235 P 8356c4df9a624e73a0f929d9aa223d3a: Deleting consensus metadata
I20260810 07:22:18.976186 5121 catalog_manager.cc:4841] TS 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): tablet a8c25fa8033243aeb74eac530f8df235 (table test-table [id=03c12fe7369444d580c193c8009c23fe]) successfully deleted
W20260810 07:22:18.981858 5120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:34753 (error 108)
I20260810 07:22:18.982131 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 5453
I20260810 07:22:18.989384 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:18.982280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34753 (local address 127.0.0.1:58318)
0810 07:22:18.989228 (+ 6948us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:18.989324 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6907}
W20260810 07:22:18.993975 5120 catalog_manager.cc:4584] TS 6abbce2e828b46e9a78ec38bc532dc86 (127.14.164.1:34753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8c25fa8033243aeb74eac530f8df235: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34753: connect: Connection refused (error 111)
I20260810 07:22:18.998663 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 5663
I20260810 07:22:19.013113 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 5093
I20260810 07:22:19.028164 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35471
--webserver_interface=127.14.164.62
--webserver_port=38499
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:19.116195 5920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:19.116423 5920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:19.116446 5920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:19.117611 5920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:19.117657 5920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:19.117672 5920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:19.117686 5920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:19.117698 5920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:19.119218 5920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=38499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:19.119434 5920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:19.119628 5920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:19.125056 5925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:19.125049 5926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:19.125267 5920 server_base.cc:1034] running on GCE node
W20260810 07:22:19.125049 5928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:19.125634 5920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:19.125946 5920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:19.127134 5920 hybrid_clock.cc:648] HybridClock initialized: now 1786346539127117 us; error 31 us; skew 500 ppm
Aug 10 07:22:19 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346539, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:19.129940 5920 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:19.130453 5920 webserver.cc:460] Webserver started at http://127.14.164.62:38499/ using document root <none> and password file <none>
I20260810 07:22:19.130683 5920 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:19.130728 5920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:19.132126 5920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:19.132872 5940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.133190 5920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:19.133312 5920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "76cd9be632a4280f974d26adad6c9f2e"
server_key_iv: "520eaa813d9de19cadc132f321db4b2d"
server_key_version: "encryptionkey@0"
I20260810 07:22:19.133711 5920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:19.152107 5920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:19.154325 5920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:19.154537 5920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:19.159071 5920 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:35471
I20260810 07:22:19.159107 6028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:35471 every 8 connection(s)
I20260810 07:22:19.159564 5920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:19.161363 6030 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:19.161996 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap starting.
I20260810 07:22:19.164999 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 5920
I20260810 07:22:19.165333 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:34753
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:35471
--webserver_port=42699
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:19.172001 6030 log.cc:824] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:19.174458 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:19.174805 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33: Bootstrap complete.
I20260810 07:22:19.177274 6030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:19.177518 6030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0bd29f032548be8dc8ac7eb6ff5f33, State: Initialized, Role: FOLLOWER
I20260810 07:22:19.177700 6030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:19.177763 6030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:19.177829 6030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:19.177879 6030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:22:19.178926 6030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:19.179047 6030 leader_election.cc:302] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a0bd29f032548be8dc8ac7eb6ff5f33; no voters:
I20260810 07:22:19.179286 6030 leader_election.cc:288] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:22:19.179327 6039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:22:19.179450 6039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [term 3 LEADER]: Becoming Leader. State: Replica: 4a0bd29f032548be8dc8ac7eb6ff5f33, State: Running, Role: LEADER
I20260810 07:22:19.179541 6039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } }
I20260810 07:22:19.179579 6030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:19.179811 6039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a0bd29f032548be8dc8ac7eb6ff5f33. Latest consensus state: current_term: 3 leader_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } } }
I20260810 07:22:19.179891 6039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:19.179813 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0bd29f032548be8dc8ac7eb6ff5f33" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35471 } } }
I20260810 07:22:19.180073 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0bd29f032548be8dc8ac7eb6ff5f33 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:19.180109 6046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:19.181077 6046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:19.181205 6046 catalog_manager.cc:1259] Loaded cluster ID: b8834e0ed5714912bc5b3a15f49d94d4
I20260810 07:22:19.181421 6046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 07:22:19.181782 6046 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe3b11759ad google::LogMessage::Fail() at ??:0
@ 0x7fe3b117888a google::LogMessage::SendToLog() at ??:0
@ 0x7fe3b11754de google::LogMessage::Flush() at ??:0
@ 0x7fe3b1176a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe3b90960f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe3b909f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe3b30430d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe3b3038954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe3b3e9c6db start_thread at ??:0
@ 0x7fe3afc7361f clone at ??:0
W20260810 07:22:19.277923 6036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:19.278191 6036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:19.278235 6036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:19.279603 6036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:19.279661 6036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:19.279691 6036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:19.281350 6036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:34753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=42699
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:19.281626 6036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:19.281970 6036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:19.288504 6036 server_base.cc:1034] running on GCE node
W20260810 07:22:19.288483 6087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:19.288647 6091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:19.288471 6085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:19.288964 6036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:19.289243 6036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:19.290531 6036 hybrid_clock.cc:648] HybridClock initialized: now 1786346539290505 us; error 35 us; skew 500 ppm
Aug 10 07:22:19 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346539, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:19.293419 6036 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:19.293934 6036 webserver.cc:460] Webserver started at http://127.14.164.1:42699/ using document root <none> and password file <none>
I20260810 07:22:19.294212 6036 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:19.294262 6036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:19.295758 6036 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:19.296605 6099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.296864 6036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:19.296965 6036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "6abbce2e828b46e9a78ec38bc532dc86"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "37021986e815b7a35343af958facfaef"
server_key_iv: "71bea22e951f0dbd62909bff02acbe7b"
server_key_version: "encryptionkey@0"
I20260810 07:22:19.297380 6036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:19.318387 6036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:19.321017 6036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:19.321203 6036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:19.321498 6036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:19.322003 6110 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:19.324218 6036 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:19.324297 6036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:22:19.324349 6036 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:19.325161 6036 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:19.325224 6036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:19.325311 6110 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap starting.
I20260810 07:22:19.332979 6036 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:34753
I20260810 07:22:19.333125 6283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:34753 every 8 connection(s)
I20260810 07:22:19.333436 6036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:19.338389 6110 log.cc:824] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:19.341763 6110 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:19.342193 6110 tablet_bootstrap.cc:492] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Bootstrap complete.
I20260810 07:22:19.342971 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 6036
I20260810 07:22:19.343286 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:38185
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:35471
--webserver_port=36807
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:19.343797 6110 ts_tablet_manager.cc:1397] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.010s
I20260810 07:22:19.345654 6110 raft_consensus.cc:348] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:19.346046 6110 raft_consensus.cc:729] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abbce2e828b46e9a78ec38bc532dc86, State: Initialized, Role: FOLLOWER
I20260810 07:22:19.346215 6110 consensus_queue.cc:260] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:19.346542 6110 ts_tablet_manager.cc:1428] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20260810 07:22:19.405553 6286 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:22:19.334209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:41045)
0810 07:22:19.334390 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.334448 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:19.334920 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:19.405331 (+ 70411us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
W20260810 07:22:19.405792 6284 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:35471 (0 consecutive failures): Network error: Failed to ping master at 127.14.164.62:35471: Client connection negotiation failed: client connection to 127.14.164.62:35471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:22:19.406168 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.406040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:50531)
0810 07:22:19.406099 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.406147 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:22:19.406564 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.406473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:48859)
0810 07:22:19.406517 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.406541 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:22:19.406718 6284 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:22:19.445551 6292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:19.445812 6292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:19.445839 6292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:19.447170 6292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:19.447213 6292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:19.447239 6292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:19.448694 6292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:38185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=36807
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:19.448973 6292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:19.449214 6292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:19.455166 6304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:19.455108 6307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:19.455116 6303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:19.455159 6292 server_base.cc:1034] running on GCE node
I20260810 07:22:19.455488 6292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:19.455742 6292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:19.456902 6292 hybrid_clock.cc:648] HybridClock initialized: now 1786346539456884 us; error 34 us; skew 500 ppm
Aug 10 07:22:19 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346539, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:19.459649 6292 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:19.460233 6292 webserver.cc:460] Webserver started at http://127.14.164.2:36807/ using document root <none> and password file <none>
I20260810 07:22:19.460479 6292 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:19.460527 6292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:19.462471 6292 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:19.463240 6319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.463593 6292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:19.463685 6292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "e4f125ea4ab64db6be83ed7566725bdb"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "73ba0a123cf2bd1348472b6607cf737f"
server_key_iv: "d25d20898cb86eca953e00454625f082"
server_key_version: "encryptionkey@0"
I20260810 07:22:19.464113 6292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:19.482074 6292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:19.484249 6292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:19.484407 6292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:19.484676 6292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:19.485080 6292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:19.485126 6292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.485149 6292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:19.485162 6292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.491736 6292 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:38185
I20260810 07:22:19.491839 6500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:38185 every 8 connection(s)
I20260810 07:22:19.492213 6292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:19.493387 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.492971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:59997)
0810 07:22:19.493139 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.493225 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
W20260810 07:22:19.493832 6501 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:35471 (0 consecutive failures): Network error: Failed to ping master at 127.14.164.62:35471: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
I20260810 07:22:19.494140 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.494046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:47169)
0810 07:22:19.494091 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.494119 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:22:19.494423 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.494336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:39701)
0810 07:22:19.494387 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.494409 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:22:19.494546 6501 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:22:19.501425 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 6292
I20260810 07:22:19.501658 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:34999
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:35471
--webserver_port=40047
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:19.588512 6504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:19.588788 6504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:19.588829 6504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:19.590029 6504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:19.590076 6504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:19.590103 6504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:19.591560 6504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:34999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=40047
--tserver_master_addrs=127.14.164.62:35471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:19.591770 6504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:19.591969 6504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:19.597703 6511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:19.597792 6504 server_base.cc:1034] running on GCE node
W20260810 07:22:19.597867 6513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:19.597703 6510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:19.598212 6504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:19.598564 6504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:19.599757 6504 hybrid_clock.cc:648] HybridClock initialized: now 1786346539599736 us; error 46 us; skew 500 ppm
Aug 10 07:22:19 dist-test-slave-x5fp krb5kdc[4223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346539, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:19.602851 6504 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:19.603401 6504 webserver.cc:460] Webserver started at http://127.14.164.3:40047/ using document root <none> and password file <none>
I20260810 07:22:19.603665 6504 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:19.603745 6504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:19.605415 6504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:22:19.606245 6522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.606591 6504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:19.606741 6504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "8356c4df9a624e73a0f929d9aa223d3a"
format_stamp: "Formatted at 2026-08-10 07:22:17 on dist-test-slave-x5fp"
server_key: "be0a58e332249c7690c343544cc5f4cd"
server_key_iv: "3f0d8d3ce6a84d63d38eac3a61d00481"
server_key_version: "encryptionkey@0"
I20260810 07:22:19.607143 6504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:19.616577 6504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:19.618975 6504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:19.619163 6504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:19.619457 6504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:19.619796 6504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:19.619856 6504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.619902 6504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:19.619930 6504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:19.626540 6504 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:34999
I20260810 07:22:19.626601 6694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:34999 every 8 connection(s)
I20260810 07:22:19.627019 6504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:19.628233 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 6504
I20260810 07:22:19.628917 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.627839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:57715)
0810 07:22:19.628178 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.628296 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":243,"threads_started":1}
W20260810 07:22:19.629256 6697 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:35471 (0 consecutive failures): Network error: Failed to ping master at 127.14.164.62:35471: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
I20260810 07:22:19.629617 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.629484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:53137)
0810 07:22:19.629545 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.629577 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:22:19.629971 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:19.629871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:42601)
0810 07:22:19.629915 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:19.629948 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 07:22:19.630110 6697 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:22:20.407512 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.407214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:59749)
0810 07:22:20.407384 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:20.407436 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260810 07:22:20.495117 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.494876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:40483)
0810 07:22:20.495020 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:20.495059 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260810 07:22:20.630745 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.630493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:44261)
0810 07:22:20.630639 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:20.630684 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260810 07:22:20.893114 6708 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:20.893266 6708 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:20.893743 6708 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
Aug 10 07:22:20 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346539, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.3@KRBTEST.COM
Aug 10 07:22:20 dist-test-slave-x5fp krb5kdc[4223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346539, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.2@KRBTEST.COM
I20260810 07:22:20.910687 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.894287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:44923 (local address 127.14.164.3:34999)
0810 07:22:20.894438 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:20.894445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:20.894487 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:22:20.895103 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:20.895105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:20.895154 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:20.895327 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:20.896480 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.897965 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.899747 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.900441 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.900750 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:20.905200 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:20.905205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:20.905205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:20.905218 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:20.907656 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:20.908410 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:20.908419 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:20.908419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:20.908813 (+ 394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:20.909331 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:20.909332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:20.909333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:20.909743 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:20.909841 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:20.910372 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 07:22:20.910544 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260810 07:22:20.910673 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.894026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:38185 (local address 127.14.164.1:37823)
0810 07:22:20.894141 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:20.894156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:20.894240 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:20.895424 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:20.895428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:20.895987 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:20.896277 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:20.896285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.898120 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.898121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:20.899562 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:20.899567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.900697 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.900698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:20.900886 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:20.901758 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:20.901771 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:20.904966 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:20.907681 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:20.907695 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:20.907698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:20.908187 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:20.908911 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:20.908912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:20.908913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:20.909179 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:20.909851 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:20.909861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:20.910143 (+ 282us) client_negotiation.cc:769] Sending connection context
0810 07:22:20.910311 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 07:22:20.910556 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":113}
I20260810 07:22:20.910817 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.894048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:34999 (local address 127.14.164.1:44923)
0810 07:22:20.894260 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:20.894272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:20.894342 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:20.895424 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:20.895428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:20.895987 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:20.896277 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:20.896285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.898123 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.898124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:20.899570 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:20.899572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.900694 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.900696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:20.900886 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:20.901761 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:20.901771 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:20.904554 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:20.907745 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:20.907746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:20.907746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:20.908187 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:20.908911 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:20.908912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:20.908913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:20.909178 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:20.910001 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:20.910003 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:20.910124 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:22:20.910330 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 07:22:20.910675 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":242,"thread_start_us":109,"threads_started":1}
I20260810 07:22:20.910806 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:20.894230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37823 (local address 127.14.164.2:38185)
0810 07:22:20.894377 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:20.894390 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:20.894454 (+ 64us) server_negotiation.cc:407] Connection header received
0810 07:22:20.895084 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:20.895088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:20.895154 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:20.895327 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:20.896470 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.897965 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.899746 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:20.900470 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:20.900753 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:20.905198 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:20.905203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:20.905203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:20.905218 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:20.907552 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:20.908410 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:20.908419 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:20.908419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:20.908792 (+ 373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:20.909331 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:20.909332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:20.909333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:20.909736 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:20.909852 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:20.910536 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 07:22:20.910672 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
W20260810 07:22:20.911834 6102 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
W20260810 07:22:20.911953 6100 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:20.911998 6100 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86; no voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb
I20260810 07:22:20.912124 6708 raft_consensus.cc:2726] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:22:21.408360 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:21.408166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:47939)
0810 07:22:21.408263 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:21.408313 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 07:22:21.495877 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:21.495610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:51539)
0810 07:22:21.495761 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:21.495828 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260810 07:22:21.631656 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:21.631398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:45095)
0810 07:22:21.631557 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:21.631605 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260810 07:22:22.409277 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:22.408999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:44243)
0810 07:22:22.409166 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:22.409213 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260810 07:22:22.496709 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:22.496446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:48651)
0810 07:22:22.496611 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:22.496657 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260810 07:22:22.632454 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:22.632205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:52547)
0810 07:22:22.632352 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:22.632395 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260810 07:22:22.729732 6721 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:22.729931 6721 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:22.730158 6721 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
W20260810 07:22:22.730697 6102 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
W20260810 07:22:22.730919 6100 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:22.731000 6100 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86; no voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb
I20260810 07:22:22.731081 6721 raft_consensus.cc:2726] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:22:23.410065 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:23.409826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:37363)
0810 07:22:23.409969 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:23.410013 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260810 07:22:23.497722 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:23.497469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:57867)
0810 07:22:23.497615 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:23.497673 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260810 07:22:23.633270 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:23.633033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:56861)
0810 07:22:23.633171 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:23.633219 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260810 07:22:24.411011 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:24.410717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:53227)
0810 07:22:24.410913 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:24.410960 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":124,"threads_started":1}
I20260810 07:22:24.498683 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:24.498420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:35927)
0810 07:22:24.498577 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:24.498629 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260810 07:22:24.534298 6734 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:24.534427 6734 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:24.534682 6734 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
W20260810 07:22:24.535151 6102 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
W20260810 07:22:24.535245 6100 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:24.535348 6100 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86; no voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb
I20260810 07:22:24.535434 6734 raft_consensus.cc:2726] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:22:24.634150 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:24.633848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:53237)
0810 07:22:24.634025 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:24.634079 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260810 07:22:25.411819 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:25.411545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:59989)
0810 07:22:25.411710 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:25.411766 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260810 07:22:25.499464 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:25.499221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:38795)
0810 07:22:25.499361 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:25.499412 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260810 07:22:25.634996 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:25.634697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:45909)
0810 07:22:25.634890 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:25.634936 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260810 07:22:26.412797 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:26.412405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:34011)
0810 07:22:26.412623 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:26.412671 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":123,"threads_started":1}
I20260810 07:22:26.500412 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:26.500095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:58671)
0810 07:22:26.500301 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:26.500355 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
I20260810 07:22:26.635869 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:26.635590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:53377)
0810 07:22:26.635773 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:26.635817 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260810 07:22:27.383708 6746 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:27.383855 6746 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:27.384094 6746 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
W20260810 07:22:27.384785 6100 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
W20260810 07:22:27.384917 6102 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:27.385005 6102 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86; no voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb
I20260810 07:22:27.385095 6746 raft_consensus.cc:2726] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:22:27.413573 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:27.413310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:35663)
0810 07:22:27.413474 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:27.413520 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260810 07:22:27.501423 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:27.501132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:48097)
0810 07:22:27.501322 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:27.501370 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260810 07:22:27.636840 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:27.636520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:57979)
0810 07:22:27.636709 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:27.636780 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260810 07:22:28.414340 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:28.414070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:57559)
0810 07:22:28.414245 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:28.414289 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260810 07:22:28.502220 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:28.501971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:44751)
0810 07:22:28.502127 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:28.502170 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260810 07:22:28.637614 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:28.637373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:34643)
0810 07:22:28.637521 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:28.637564 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260810 07:22:29.415187 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:29.414931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:43797)
0810 07:22:29.415087 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:29.415136 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260810 07:22:29.503057 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:29.502803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:52185)
0810 07:22:29.502959 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:29.503003 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260810 07:22:29.638614 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:29.638331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:46543)
0810 07:22:29.638513 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:29.638560 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260810 07:22:30.270779 6760 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:30.270888 6760 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:30.271124 6760 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
W20260810 07:22:30.271572 6100 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
W20260810 07:22:30.271673 6102 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:30.271739 6102 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86; no voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb
I20260810 07:22:30.271827 6760 raft_consensus.cc:2726] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:22:30.416071 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:30.415753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:52593)
0810 07:22:30.415971 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:30.416017 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
I20260810 07:22:30.503938 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:30.503624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:37097)
0810 07:22:30.503842 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:30.503886 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
I20260810 07:22:30.639526 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:30.639246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:45385)
0810 07:22:30.639433 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:30.639474 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260810 07:22:31.416917 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:31.416614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:54949)
0810 07:22:31.416799 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:31.416846 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":68,"threads_started":1}
I20260810 07:22:31.504920 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:31.504497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:42103)
0810 07:22:31.504660 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:31.504703 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":64,"threads_started":1}
I20260810 07:22:31.640367 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:31.640122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:55601)
0810 07:22:31.640277 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:31.640317 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260810 07:22:32.417770 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:32.417484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:43947)
0810 07:22:32.417664 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:32.417712 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260810 07:22:32.505652 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:32.505418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:48839)
0810 07:22:32.505557 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:32.505597 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260810 07:22:32.641328 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:32.641054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:50767)
0810 07:22:32.641206 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:32.641262 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260810 07:22:33.418725 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.418443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:37497)
0810 07:22:33.418615 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:33.418669 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260810 07:22:33.506567 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.506264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:47881)
0810 07:22:33.506464 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:33.506513 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260810 07:22:33.642164 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.641865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:45517)
0810 07:22:33.642062 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:33.642112 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
I20260810 07:22:34.419701 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.419446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.1:54173)
0810 07:22:34.419604 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:34.419652 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260810 07:22:34.507534 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.507241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.2:48055)
0810 07:22:34.507425 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:34.507480 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260810 07:22:34.557498 6786 raft_consensus.cc:482] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:34.557632 6786 raft_consensus.cc:504] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8356c4df9a624e73a0f929d9aa223d3a" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 34999 } } peers { permanent_uuid: "e4f125ea4ab64db6be83ed7566725bdb" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 38185 } } peers { permanent_uuid: "6abbce2e828b46e9a78ec38bc532dc86" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34753 } }
I20260810 07:22:34.557940 6786 leader_election.cc:288] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999), e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185)
W20260810 07:22:34.558535 6100 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4f125ea4ab64db6be83ed7566725bdb (127.14.164.2:38185): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
W20260810 07:22:34.558624 6102 leader_election.cc:341] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8356c4df9a624e73a0f929d9aa223d3a (127.14.164.3:34999): Not found: Tablet not found: a8c25fa8033243aeb74eac530f8df235
I20260810 07:22:34.558699 6102 leader_election.cc:302] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6abbce2e828b46e9a78ec38bc532dc86; no voters: 8356c4df9a624e73a0f929d9aa223d3a, e4f125ea4ab64db6be83ed7566725bdb
I20260810 07:22:34.558861 6786 raft_consensus.cc:2726] T a8c25fa8033243aeb74eac530f8df235 P 6abbce2e828b46e9a78ec38bc532dc86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:22:34.640079 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 6036
I20260810 07:22:34.644989 6787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.643219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35471 (local address 127.14.164.3:45723)
0810 07:22:34.644819 (+ 1600us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:34.644894 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1576,"thread_start_us":1480,"threads_started":1}
I20260810 07:22:34.655149 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 6292
I20260810 07:22:34.670603 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 6504
2026-08-10T07:22:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19660 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:35 dist-test-slave-x5fp krb5kdc[6790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:35 dist-test-slave-x5fp krb5kdc[6790](info): set up 2 sockets
Aug 10 07:22:35 dist-test-slave-x5fp krb5kdc[6790](info): commencing operation
krb5kdc: starting...
W20260810 07:22:36.510329 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:22:36 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:22:36.652336 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42923
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:36.741516 6812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:36.741747 6812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:36.741771 6812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:36.742990 6812 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:22:36.743036 6812 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:22:36.743047 6812 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:22:36.743059 6812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:36.743072 6812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:36.743129 6812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:36.743146 6812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:36.743162 6812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:36.744769 6812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:36.745067 6812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:36.745340 6812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:36.750837 6820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:36.750828 6827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:36.750968 6812 server_base.cc:1034] running on GCE node
W20260810 07:22:36.750828 6819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:36.751453 6812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:36.751771 6812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:36.752925 6812 hybrid_clock.cc:648] HybridClock initialized: now 1786346556752915 us; error 34 us; skew 500 ppm
Aug 10 07:22:36 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346556, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:36.756517 6812 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:36.757153 6812 webserver.cc:460] Webserver started at http://127.14.164.62:42481/ using document root <none> and password file <none>
I20260810 07:22:36.757476 6812 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:36.757588 6812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:36.757858 6812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:36.759475 6812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "efd43fc9ea2c42d6a200c41ae4143ed1"
format_stamp: "Formatted at 2026-08-10 07:22:36 on dist-test-slave-x5fp"
server_key: "98f84e9f44610069690e2d057e47c77d"
server_key_iv: "32ab84e83e9c1a883bb3d78d04d8ffc2"
server_key_version: "encryptionkey@0"
I20260810 07:22:36.760075 6812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "efd43fc9ea2c42d6a200c41ae4143ed1"
format_stamp: "Formatted at 2026-08-10 07:22:36 on dist-test-slave-x5fp"
server_key: "98f84e9f44610069690e2d057e47c77d"
server_key_iv: "32ab84e83e9c1a883bb3d78d04d8ffc2"
server_key_version: "encryptionkey@0"
I20260810 07:22:36.763406 6812 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:22:36.765419 6835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:36.765849 6812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:36.765981 6812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "efd43fc9ea2c42d6a200c41ae4143ed1"
format_stamp: "Formatted at 2026-08-10 07:22:36 on dist-test-slave-x5fp"
server_key: "98f84e9f44610069690e2d057e47c77d"
server_key_iv: "32ab84e83e9c1a883bb3d78d04d8ffc2"
server_key_version: "encryptionkey@0"
I20260810 07:22:36.766081 6812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:36.774830 6812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:36.777501 6812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:36.777670 6812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:36.782627 6812 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42923
I20260810 07:22:36.782702 6918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42923 every 8 connection(s)
I20260810 07:22:36.783092 6812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:36.784257 6920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:36.787966 6920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap starting.
I20260810 07:22:36.788849 6920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:36.788947 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 6812
I20260810 07:22:36.789072 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:22:36.789326 14992 external_mini_cluster.cc:1428] Setting key b2d264b56e4b2a434324072f546ded57
I20260810 07:22:36.789526 6920 log.cc:824] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:36.790511 6920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: No bootstrap required, opened a new log
I20260810 07:22:36.793630 6920 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:36.793846 6920 raft_consensus.cc:374] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:36.793893 6920 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd43fc9ea2c42d6a200c41ae4143ed1, State: Initialized, Role: FOLLOWER
I20260810 07:22:36.794073 6920 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:36.794174 6920 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:36.794250 6920 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:36.794312 6920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:36.795184 6920 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:36.795308 6920 leader_election.cc:302] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd43fc9ea2c42d6a200c41ae4143ed1; no voters:
I20260810 07:22:36.795594 6920 leader_election.cc:288] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:36.795657 6927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:36.795917 6927 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 LEADER]: Becoming Leader. State: Replica: efd43fc9ea2c42d6a200c41ae4143ed1, State: Running, Role: LEADER
I20260810 07:22:36.796000 6920 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:36.796069 6927 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:36.796581 6928 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } } }
I20260810 07:22:36.796604 6930 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd43fc9ea2c42d6a200c41ae4143ed1. Latest consensus state: current_term: 1 leader_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } } }
I20260810 07:22:36.796787 6928 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:36.796810 6930 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:36.797226 6937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:36.797771 6937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:22:36 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:36.802165 6937 catalog_manager.cc:1347] Generated new cluster ID: 86243680af6e441d84ddecebb7aafa39
I20260810 07:22:36.802224 6937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:36.803066 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.789998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.14.164.62:42923)
0810 07:22:36.790168 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.790178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.790221 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:22:36.790968 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.790978 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.791050 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.791212 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:36.792069 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.793411 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.794434 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.795215 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.795341 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.798981 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:36.798985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:36.798985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:36.798999 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:36.801025 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:36.801492 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:36.801496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:36.801497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:36.801614 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:36.801843 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:36.801844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:36.801844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:36.802169 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:36.802352 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.802729 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.802862 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260810 07:22:36.831771 6937 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:36.832581 6937 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:36.838224 6937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Generated new TSK 0
I20260810 07:22:36.838477 6937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:22:36.882324 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:42923
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:37.006114 6954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:37.006392 6954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:37.006424 6954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:37.007627 6954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:37.007675 6954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:37.007704 6954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:37.009433 6954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:37.009716 6954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:37.009974 6954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:37.015511 6960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:37.015502 6965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:37.015504 6961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:37.016585 6954 server_base.cc:1034] running on GCE node
I20260810 07:22:37.016880 6954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:37.017201 6954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:37.018415 6954 hybrid_clock.cc:648] HybridClock initialized: now 1786346557018394 us; error 38 us; skew 500 ppm
Aug 10 07:22:37 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346557, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:37.028537 6954 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:37.029125 6954 webserver.cc:460] Webserver started at http://127.14.164.1:46495/ using document root <none> and password file <none>
I20260810 07:22:37.029424 6954 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:37.029496 6954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:37.029654 6954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:37.030812 6954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "3d9bca5957374235a6ed42d64be7617a"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "5787a387650119c4419c77be15746ded"
server_key_iv: "a22983931eee24ba6339ca29bca5e78e"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.031198 6954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "3d9bca5957374235a6ed42d64be7617a"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "5787a387650119c4419c77be15746ded"
server_key_iv: "a22983931eee24ba6339ca29bca5e78e"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.032691 6954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:22:37.033711 6976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.034044 6954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:37.034168 6954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "3d9bca5957374235a6ed42d64be7617a"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "5787a387650119c4419c77be15746ded"
server_key_iv: "a22983931eee24ba6339ca29bca5e78e"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.034255 6954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:37.064534 6954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:37.067374 6954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:37.067569 6954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:37.067829 6954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:37.068255 6954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:37.068311 6954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.068353 6954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:37.068377 6954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.075697 6954 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:34815
I20260810 07:22:37.075743 7162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:34815 every 8 connection(s)
I20260810 07:22:37.076122 6954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:37.081079 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 6954
I20260810 07:22:37.081219 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:22:37.081461 14992 external_mini_cluster.cc:1428] Setting key 7dad89ad4f2b33ee6bb65d943f5e47c7
Aug 10 07:22:37 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346557, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:37.087555 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.077018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:51513 (local address 127.14.164.62:42923)
0810 07:22:37.077096 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.077098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.077312 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:22:37.077863 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.077865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.077879 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.077963 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:37.078973 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.079553 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.080561 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.080861 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.080943 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.083855 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:37.083857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:37.083858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:37.083868 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:37.085704 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.086233 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.086235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.086235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.086302 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.086634 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.086635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.086635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.086827 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:37.086877 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.087296 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.087424 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:22:37.088186 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.076983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:51513)
0810 07:22:37.077167 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.077212 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.077725 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.078038 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.078042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.078655 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:37.078863 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.078867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.079644 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.079644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.080469 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.080471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.080968 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.080969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.081135 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.081791 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:37.081803 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:37.083730 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:37.085806 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:37.085815 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:37.085818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:37.086134 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:37.086380 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:37.086381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:37.086381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:37.086543 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:37.086933 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:37.086941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:37.087118 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.087788 (+ 670us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.087961 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260810 07:22:37.088943 7163 heartbeater.cc:347] Connected to a master server at 127.14.164.62:42923
I20260810 07:22:37.089105 7163 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:37.089334 7163 heartbeater.cc:511] Master 127.14.164.62:42923 requested a full tablet report, sending...
I20260810 07:22:37.090175 6861 ts_manager.cc:194] Registered new tserver with Master: 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815)
I20260810 07:22:37.090803 6861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:51513
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:22:37.112890 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:42923
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:37.210111 7171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:37.210321 7171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:37.210347 7171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:37.211477 7171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:37.211552 7171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:37.211601 7171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:37.213172 7171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:37.213460 7171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:37.213717 7171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:37.220281 7187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:37.220459 7185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:37.220312 7184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:37.220419 7171 server_base.cc:1034] running on GCE node
I20260810 07:22:37.220834 7171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:37.221109 7171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:37.222311 7171 hybrid_clock.cc:648] HybridClock initialized: now 1786346557222293 us; error 32 us; skew 500 ppm
Aug 10 07:22:37 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346557, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:37.225270 7171 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:37.225807 7171 webserver.cc:460] Webserver started at http://127.14.164.2:36227/ using document root <none> and password file <none>
I20260810 07:22:37.226085 7171 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:37.226161 7171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:37.226336 7171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:37.227445 7171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "20c8afa7ed8b4918856791fb369e4456"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "3ae56a245f1462ba2735c77efa0b3591"
server_key_iv: "960b36fe1e18327629e0d67c061e1c06"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.227833 7171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "20c8afa7ed8b4918856791fb369e4456"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "3ae56a245f1462ba2735c77efa0b3591"
server_key_iv: "960b36fe1e18327629e0d67c061e1c06"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.229485 7171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:22:37.230410 7197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.230630 7171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:37.230782 7171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "20c8afa7ed8b4918856791fb369e4456"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "3ae56a245f1462ba2735c77efa0b3591"
server_key_iv: "960b36fe1e18327629e0d67c061e1c06"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.230851 7171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:37.251837 7171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:37.254604 7171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:37.254791 7171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:37.255075 7171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:37.255410 7171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:37.255468 7171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.255511 7171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:37.255553 7171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.262213 7171 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:45255
I20260810 07:22:37.262271 7377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:45255 every 8 connection(s)
I20260810 07:22:37.262605 7171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:22:37 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346557, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:37.270347 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 7171
I20260810 07:22:37.270503 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:22:37.270750 14992 external_mini_cluster.cc:1428] Setting key 10cf400e753e48900d1fed54d0211fbb
I20260810 07:22:37.273068 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.263439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:54489 (local address 127.14.164.62:42923)
0810 07:22:37.263487 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.263489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.263703 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:22:37.264305 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.264306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.264321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.264380 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:37.265526 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.266060 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.267119 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.267336 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.267431 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.269769 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:37.269771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:37.269771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:37.269780 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:37.271590 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.271995 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.271996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.271997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.272063 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.272328 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.272329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.272329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.272464 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:37.272536 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.272878 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.272976 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:22:37.273274 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.263409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:54489)
0810 07:22:37.263572 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.263611 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.264188 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.264546 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.264550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.265212 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:37.265417 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.265422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.266177 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.266178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.267021 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.267023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.267441 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.267442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.267571 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.268139 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:37.268147 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:37.269657 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:37.271664 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:37.271671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:37.271675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:37.271908 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:37.272130 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:37.272131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:37.272131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:37.272249 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:37.272540 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:37.272548 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:37.272693 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.272846 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.273014 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260810 07:22:37.274231 7379 heartbeater.cc:347] Connected to a master server at 127.14.164.62:42923
I20260810 07:22:37.274354 7379 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:37.274561 7379 heartbeater.cc:511] Master 127.14.164.62:42923 requested a full tablet report, sending...
I20260810 07:22:37.274986 6861 ts_manager.cc:194] Registered new tserver with Master: 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255)
I20260810 07:22:37.275512 6861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:54489
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:22:37.304908 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:42923
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:37.427597 7388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:37.427927 7388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:37.427960 7388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:37.429601 7388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:37.429677 7388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:37.429718 7388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:37.431978 7388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:37.432332 7388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:37.432646 7388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:37.438441 7396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:37.438500 7395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:37.438495 7402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:37.438823 7388 server_base.cc:1034] running on GCE node
I20260810 07:22:37.439010 7388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:37.439213 7388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:37.440389 7388 hybrid_clock.cc:648] HybridClock initialized: now 1786346557440372 us; error 34 us; skew 500 ppm
Aug 10 07:22:37 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346557, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:37.443735 7388 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:37.444180 7388 webserver.cc:460] Webserver started at http://127.14.164.3:42015/ using document root <none> and password file <none>
I20260810 07:22:37.444442 7388 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:37.444515 7388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:37.444674 7388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:37.446076 7388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "671dd9929f9e44d9822a1510da876fb6"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "d10d01a5511624fdfadcc3b54a8ba0ef"
server_key_iv: "9538e39faedbafc523ff8a7ddc589310"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.446581 7388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "671dd9929f9e44d9822a1510da876fb6"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "d10d01a5511624fdfadcc3b54a8ba0ef"
server_key_iv: "9538e39faedbafc523ff8a7ddc589310"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.448230 7388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:22:37.449307 7414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.449496 7388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:37.449589 7388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "671dd9929f9e44d9822a1510da876fb6"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "d10d01a5511624fdfadcc3b54a8ba0ef"
server_key_iv: "9538e39faedbafc523ff8a7ddc589310"
server_key_version: "encryptionkey@0"
I20260810 07:22:37.449648 7388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:37.466130 7388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:37.469210 7388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:37.469377 7388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:37.469803 7388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:37.470314 7388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:37.470359 7388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.470383 7388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:37.470398 7388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:37.478037 7388 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:40099
I20260810 07:22:37.478127 7590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:40099 every 8 connection(s)
I20260810 07:22:37.478610 7388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:37.482793 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 7388
I20260810 07:22:37.482999 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:22:37.483245 14992 external_mini_cluster.cc:1428] Setting key fb272b8f7b3c0ed7d0f6e99f60a18ac5
Aug 10 07:22:37 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346557, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:37.491938 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.479610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:59709)
0810 07:22:37.479851 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.479924 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.480507 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.480837 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.480841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.481518 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:37.481751 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.481757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.483065 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.483067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.484516 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.484520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.485143 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.485144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.485289 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.485995 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:37.486005 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:37.487636 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:37.490241 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:37.490248 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:37.490251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:37.490500 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:37.490777 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:37.490778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:37.490778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:37.490905 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:37.491278 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:37.491286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:37.491440 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.491551 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.491743 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":150,"threads_started":1}
I20260810 07:22:37.491957 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.479625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:59709 (local address 127.14.164.62:42923)
0810 07:22:37.479702 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.479706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.480043 (+ 337us) server_negotiation.cc:407] Connection header received
0810 07:22:37.480615 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.480617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.480629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.480693 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:37.481883 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.482796 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.484612 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.485016 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.485105 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.487745 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:37.487747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:37.487747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:37.487757 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:37.490085 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.490589 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.490590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.490590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.490667 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.490990 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.490991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.490992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.491172 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:37.491256 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.491602 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.491696 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:22:37.493129 7591 heartbeater.cc:347] Connected to a master server at 127.14.164.62:42923
I20260810 07:22:37.493257 7591 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:37.493479 7591 heartbeater.cc:511] Master 127.14.164.62:42923 requested a full tablet report, sending...
I20260810 07:22:37.493937 6861 ts_manager.cc:194] Registered new tserver with Master: 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
I20260810 07:22:37.494385 6861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:59709
I20260810 07:22:37.495611 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:37.504966 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.497264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.14.164.62:42923)
0810 07:22:37.497352 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.497354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.497405 (+ 51us) server_negotiation.cc:407] Connection header received
0810 07:22:37.497485 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.497486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.497498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.497553 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:37.498187 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.498982 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.499799 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.500129 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.500218 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.501619 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:37.501620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:37.501621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:37.501628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:37.503376 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.503791 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.503792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.503793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.503865 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:37.504171 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:37.504173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:37.504173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:37.504407 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:37.504521 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.504803 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.504881 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:22:37.507187 6861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:37.509707 6861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:37.516948 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.512453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:45255 (local address 127.0.0.1:46054)
0810 07:22:37.512633 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.512680 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.512811 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.513396 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.513397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.513399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:37.513486 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.513487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.514420 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.514422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.515572 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.515575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.516548 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.516549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.516638 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.516654 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.516773 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.516850 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":17,"thread_start_us":115,"threads_started":1}
I20260810 07:22:37.516944 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.512545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.14.164.2:45255)
0810 07:22:37.513174 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.513179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.513192 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:22:37.513245 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.513247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.513265 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.513566 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:37.513641 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.514315 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.515674 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.516450 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.516527 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.516583 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.516808 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.516854 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":88,"threads_started":1}
I20260810 07:22:37.517269 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.512455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34815 (local address 127.0.0.1:58038)
0810 07:22:37.512803 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.512816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.512873 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.513348 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.513351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.513354 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:37.513439 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.513442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.514946 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.514948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.515968 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.515972 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.516848 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.516849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.516926 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.516930 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.516965 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.517204 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":83,"thread_start_us":81,"threads_started":1}
I20260810 07:22:37.517359 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.512700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:40099 (local address 127.0.0.1:41684)
0810 07:22:37.512934 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.512946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.512988 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.513446 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.513447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.513448 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:37.513484 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.513486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.514915 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.514916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.516020 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.516023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.517014 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.517015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.517087 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.517094 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.517123 (+ 29us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.517247 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":66,"thread_start_us":111,"threads_started":1}
I20260810 07:22:37.517658 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.513008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.14.164.3:40099)
0810 07:22:37.513267 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.513271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.513282 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:37.513343 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.513346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.513362 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.513450 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:37.513538 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.514792 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.516117 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.516912 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.517173 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.517450 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.517478 (+ 28us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.517520 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260810 07:22:37.517828 7275 tablet_service.cc:1467] Processing CreateTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac (DEFAULT_TABLE table=test-table [id=d7680d4119764744b643cffddfb8ca26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:37.518074 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.512680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.14.164.1:34815)
0810 07:22:37.513056 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.513062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.513074 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:22:37.513119 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.513122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.513141 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.513228 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:37.513532 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.514789 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.516070 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.516709 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.517797 (+ 1088us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.517890 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.517918 (+ 28us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.517966 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":107,"threads_started":1}
I20260810 07:22:37.518234 7275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:37.521462 7487 tablet_service.cc:1467] Processing CreateTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac (DEFAULT_TABLE table=test-table [id=d7680d4119764744b643cffddfb8ca26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:37.521744 7616 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Bootstrap starting.
I20260810 07:22:37.521867 7487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:37.521736 7053 tablet_service.cc:1467] Processing CreateTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac (DEFAULT_TABLE table=test-table [id=d7680d4119764744b643cffddfb8ca26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:37.522079 7053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:37.522370 7616 tablet_bootstrap.cc:654] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:37.522801 7616 log.cc:824] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:37.523517 7616 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: No bootstrap required, opened a new log
I20260810 07:22:37.523566 7616 ts_tablet_manager.cc:1397] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:37.525542 7620 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Bootstrap starting.
I20260810 07:22:37.525484 7616 raft_consensus.cc:348] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.525692 7616 raft_consensus.cc:374] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:37.525748 7616 raft_consensus.cc:729] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c8afa7ed8b4918856791fb369e4456, State: Initialized, Role: FOLLOWER
I20260810 07:22:37.525982 7616 consensus_queue.cc:260] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.526364 7379 heartbeater.cc:503] Master 127.14.164.62:42923 was elected leader, sending a full tablet report...
I20260810 07:22:37.526429 7620 tablet_bootstrap.cc:654] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:37.526371 7616 ts_tablet_manager.cc:1428] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:22:37.526492 7619 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Bootstrap starting.
I20260810 07:22:37.527055 7620 log.cc:824] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:37.527920 7620 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: No bootstrap required, opened a new log
I20260810 07:22:37.527981 7620 ts_tablet_manager.cc:1397] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:22:37.529376 7619 tablet_bootstrap.cc:654] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:37.529821 7620 raft_consensus.cc:348] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.529991 7620 raft_consensus.cc:374] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:37.529995 7619 log.cc:824] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:37.530036 7620 raft_consensus.cc:729] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671dd9929f9e44d9822a1510da876fb6, State: Initialized, Role: FOLLOWER
I20260810 07:22:37.530164 7620 consensus_queue.cc:260] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.530483 7591 heartbeater.cc:503] Master 127.14.164.62:42923 was elected leader, sending a full tablet report...
I20260810 07:22:37.530472 7620 ts_tablet_manager.cc:1428] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:22:37.530848 7619 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: No bootstrap required, opened a new log
I20260810 07:22:37.530907 7619 ts_tablet_manager.cc:1397] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:22:37.533260 7619 raft_consensus.cc:348] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.533480 7619 raft_consensus.cc:374] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:37.533520 7619 raft_consensus.cc:729] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9bca5957374235a6ed42d64be7617a, State: Initialized, Role: FOLLOWER
I20260810 07:22:37.533634 7619 consensus_queue.cc:260] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.533885 7619 ts_tablet_manager.cc:1428] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:22:37.534013 7163 heartbeater.cc:503] Master 127.14.164.62:42923 was elected leader, sending a full tablet report...
W20260810 07:22:37.576963 7164 tablet.cc:2369] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:22:37.729544 7592 tablet.cc:2369] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:37.759409 7627 raft_consensus.cc:482] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:37.759541 7627 raft_consensus.cc:504] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.759923 7627 leader_election.cc:288] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815)
W20260810 07:22:37.763370 7380 tablet.cc:2369] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:37.763597 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.760139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34815 (local address 127.14.164.3:41763)
0810 07:22:37.760395 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.760403 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.760449 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.760588 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.760588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.760590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:37.760650 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.760651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.761641 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.761643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.762495 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.762497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.763313 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.763314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.763393 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.763397 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.763446 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.763487 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":100,"thread_start_us":97,"threads_started":1}
I20260810 07:22:37.763743 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.760280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:43625 (local address 127.14.164.2:45255)
0810 07:22:37.760370 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.760373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.760386 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:22:37.760419 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.760420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.760453 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.760531 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:37.760799 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.761522 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.762742 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.763337 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.763398 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.763407 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.763631 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.763680 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:22:37.763857 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.760306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:41763 (local address 127.14.164.1:34815)
0810 07:22:37.760385 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.760388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.760457 (+ 69us) server_negotiation.cc:407] Connection header received
0810 07:22:37.760486 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.760487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.760500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.760562 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:37.760767 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.761547 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.762594 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.763171 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.763251 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.763260 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.763731 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.763781 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:22:37.763906 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.760126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:45255 (local address 127.14.164.3:43625)
0810 07:22:37.760193 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.760207 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:37.760263 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:37.760556 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:37.760557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:37.760559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:37.760647 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.760649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.761629 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.761630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.762644 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:37.762647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.763436 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.763437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:37.763509 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.763511 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:22:37.763556 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 07:22:37.763588 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":26}
I20260810 07:22:37.764185 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638524d9d7034e5cb79ca0ce9dd6c2ac" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9bca5957374235a6ed42d64be7617a" is_pre_election: true
I20260810 07:22:37.764317 7306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638524d9d7034e5cb79ca0ce9dd6c2ac" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8afa7ed8b4918856791fb369e4456" is_pre_election: true
I20260810 07:22:37.764407 7087 raft_consensus.cc:2454] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 0.
I20260810 07:22:37.764448 7306 raft_consensus.cc:2454] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 0.
I20260810 07:22:37.764635 7418 leader_election.cc:302] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a, 671dd9929f9e44d9822a1510da876fb6; no voters:
I20260810 07:22:37.764814 7627 raft_consensus.cc:2781] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:37.764890 7627 raft_consensus.cc:482] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:37.764909 7627 raft_consensus.cc:3037] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:37.765772 7627 raft_consensus.cc:504] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.765933 7627 leader_election.cc:288] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 election: Requested vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815)
I20260810 07:22:37.766119 7306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638524d9d7034e5cb79ca0ce9dd6c2ac" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8afa7ed8b4918856791fb369e4456"
I20260810 07:22:37.766209 7306 raft_consensus.cc:3037] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:37.766273 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638524d9d7034e5cb79ca0ce9dd6c2ac" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9bca5957374235a6ed42d64be7617a"
I20260810 07:22:37.766373 7087 raft_consensus.cc:3037] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:37.767138 7306 raft_consensus.cc:2454] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 1.
I20260810 07:22:37.767154 7087 raft_consensus.cc:2454] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 1.
I20260810 07:22:37.767350 7417 leader_election.cc:302] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6; no voters:
I20260810 07:22:37.767460 7627 raft_consensus.cc:2781] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:37.767666 7627 raft_consensus.cc:686] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 671dd9929f9e44d9822a1510da876fb6, State: Running, Role: LEADER
I20260810 07:22:37.767791 7627 consensus_queue.cc:237] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:37.768760 6861 catalog_manager.cc:5495] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "671dd9929f9e44d9822a1510da876fb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } health_report { overall_health: UNKNOWN } } }
I20260810 07:22:37.802160 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.799127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.14.164.3:40099)
0810 07:22:37.799215 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:37.799218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:37.799229 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:37.799279 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:37.799280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:37.799300 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:37.799383 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:37.799556 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.800257 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.801244 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:37.801459 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:37.801545 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:37.801737 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:37.801857 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:37.801954 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:37.802004 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 07:22:37.802040 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:22:37.804549 7306 raft_consensus.cc:1261] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Refusing update from remote peer 671dd9929f9e44d9822a1510da876fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:37.804561 7087 raft_consensus.cc:1261] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Refusing update from remote peer 671dd9929f9e44d9822a1510da876fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:37.804982 7627 consensus_queue.cc:1035] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:37.805151 7630 consensus_queue.cc:1035] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:37.807929 7635 mvcc.cc:205] Tried to move back new op lower bound from 7316875500763721728 to 7316875500617564160. Current Snapshot: MvccSnapshot[applied={T|T < 7316875500763721728}]
I20260810 07:22:37.808705 7637 mvcc.cc:205] Tried to move back new op lower bound from 7316875500763721728 to 7316875500617564160. Current Snapshot: MvccSnapshot[applied={T|T < 7316875500763721728}]
I20260810 07:22:37.809845 7639 mvcc.cc:205] Tried to move back new op lower bound from 7316875500763721728 to 7316875500617564160. Current Snapshot: MvccSnapshot[applied={T|T < 7316875500763721728 or (T in {7316875500763721728})}]
I20260810 07:22:37.810599 6861 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:37.810709 6861 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:37.811641 6861 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Sending DeleteTablet for 3 replicas of tablet 638524d9d7034e5cb79ca0ce9dd6c2ac
I20260810 07:22:37.812060 7275 tablet_service.cc:1514] Processing DeleteTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:46054
I20260810 07:22:37.812157 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 6954
I20260810 07:22:37.812155 7053 tablet_service.cc:1514] Processing DeleteTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:58038
I20260810 07:22:37.812304 7651 tablet_replica.cc:330] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: stopping tablet replica
I20260810 07:22:37.812407 7651 raft_consensus.cc:2229] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:37.812523 7651 raft_consensus.cc:2258] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:37.815056 7487 tablet_service.cc:1514] Processing DeleteTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:41684
I20260810 07:22:37.816651 7653 tablet_replica.cc:330] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: stopping tablet replica
I20260810 07:22:37.816813 7653 raft_consensus.cc:2229] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:22:37.816951 7653 raft_consensus.cc:2258] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:37.822563 7653 ts_tablet_manager.cc:1905] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:37.823993 7651 ts_tablet_manager.cc:1905] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:37.824857 7653 ts_tablet_manager.cc:1918] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:37.824950 7653 log.cc:1196] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/638524d9d7034e5cb79ca0ce9dd6c2ac
I20260810 07:22:37.825709 7651 ts_tablet_manager.cc:1918] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:37.825724 7653 ts_tablet_manager.cc:1939] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 671dd9929f9e44d9822a1510da876fb6: Deleting consensus metadata
I20260810 07:22:37.825799 7651 log.cc:1196] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/638524d9d7034e5cb79ca0ce9dd6c2ac
I20260810 07:22:37.826093 7651 ts_tablet_manager.cc:1939] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 20c8afa7ed8b4918856791fb369e4456: Deleting consensus metadata
I20260810 07:22:37.826584 6841 catalog_manager.cc:4841] TS 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): tablet 638524d9d7034e5cb79ca0ce9dd6c2ac (table test-table [id=d7680d4119764744b643cffddfb8ca26]) successfully deleted
I20260810 07:22:37.827520 6841 catalog_manager.cc:4841] TS 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): tablet 638524d9d7034e5cb79ca0ce9dd6c2ac (table test-table [id=d7680d4119764744b643cffddfb8ca26]) successfully deleted
W20260810 07:22:37.830090 6842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:34815 (error 108)
I20260810 07:22:37.830682 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:37.830542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34815 (local address 127.0.0.1:58048)
0810 07:22:37.830601 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:37.830645 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:22:37.830802 6842 catalog_manager.cc:4584] TS 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34815: connect: Connection refused (error 111)
I20260810 07:22:37.831151 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 7171
I20260810 07:22:37.846199 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 7388
I20260810 07:22:37.858733 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 6812
I20260810 07:22:57.870980 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42923
--webserver_interface=127.14.164.62
--webserver_port=42481
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:57.958676 7655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:57.959044 7655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:57.959085 7655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:57.960937 7655 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:22:57.961058 7655 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:22:57.961094 7655 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:22:57.961118 7655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:57.961149 7655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:57.961179 7655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:57.961198 7655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:57.961229 7655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:57.963713 7655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=42481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:57.963999 7655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:57.964294 7655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:57.975476 7663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:57.975481 7662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:57.975476 7665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:57.975498 7655 server_base.cc:1034] running on GCE node
I20260810 07:22:57.976161 7655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:57.976519 7655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:57.977679 7655 hybrid_clock.cc:648] HybridClock initialized: now 1786346577977616 us; error 80 us; skew 500 ppm
Aug 10 07:22:57 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346577, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:57.986738 7655 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:22:57.987290 7655 webserver.cc:460] Webserver started at http://127.14.164.62:42481/ using document root <none> and password file <none>
I20260810 07:22:57.987581 7655 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:57.987674 7655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:57.989437 7655 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:22:57.990367 7677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:57.990645 7655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:57.990777 7655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "efd43fc9ea2c42d6a200c41ae4143ed1"
format_stamp: "Formatted at 2026-08-10 07:22:36 on dist-test-slave-x5fp"
server_key: "98f84e9f44610069690e2d057e47c77d"
server_key_iv: "32ab84e83e9c1a883bb3d78d04d8ffc2"
server_key_version: "encryptionkey@0"
I20260810 07:22:57.991259 7655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:58.014429 7655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:58.017251 7655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:58.017395 7655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:58.022109 7655 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42923
I20260810 07:22:58.022171 7759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42923 every 8 connection(s)
I20260810 07:22:58.022662 7655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:58.024451 7760 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:58.025256 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap starting.
I20260810 07:22:58.028556 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 7655
I20260810 07:22:58.028947 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:34815
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:42923
--webserver_port=46495
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:58.034945 7760 log.cc:824] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:58.037179 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:58.037520 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap complete.
I20260810 07:22:58.039902 7760 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:58.040165 7760 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd43fc9ea2c42d6a200c41ae4143ed1, State: Initialized, Role: FOLLOWER
I20260810 07:22:58.040385 7760 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:58.040486 7760 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:58.040550 7760 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:58.040601 7760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:58.041682 7760 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:58.041807 7760 leader_election.cc:302] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd43fc9ea2c42d6a200c41ae4143ed1; no voters:
I20260810 07:22:58.042033 7760 leader_election.cc:288] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:22:58.042074 7765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:22:58.042200 7765 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 LEADER]: Becoming Leader. State: Replica: efd43fc9ea2c42d6a200c41ae4143ed1, State: Running, Role: LEADER
I20260810 07:22:58.042292 7765 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:22:58.042326 7760 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:58.042604 7765 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd43fc9ea2c42d6a200c41ae4143ed1. Latest consensus state: current_term: 2 leader_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } } }
I20260810 07:22:58.042687 7765 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:58.042605 7766 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } } }
I20260810 07:22:58.042856 7766 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:58.044296 7787 catalog_manager.cc:1259] Loaded cluster ID: 86243680af6e441d84ddecebb7aafa39
I20260810 07:22:58.044359 7787 catalog_manager.cc:1552] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: loading cluster ID for follower catalog manager: success
I20260810 07:22:58.045593 7787 catalog_manager.cc:1574] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: acquiring CA information for follower catalog manager: success
I20260810 07:22:58.046015 7787 catalog_manager.cc:1602] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:22:58.046090 7769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:58.046473 7769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:58.046561 7769 catalog_manager.cc:1259] Loaded cluster ID: 86243680af6e441d84ddecebb7aafa39
I20260810 07:22:58.046597 7769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:58.046762 7769 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:58.046849 7769 catalog_manager.cc:5879] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Loaded TSK: 0
I20260810 07:22:58.058089 7769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Generated new TSK 1
I20260810 07:22:58.058193 7769 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:22:58.120266 7763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:58.120518 7763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:58.120558 7763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:58.121755 7763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:58.121824 7763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:58.121879 7763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:22:58.123389 7763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:34815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=46495
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:58.123675 7763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:58.123930 7763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:58.130195 7798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:58.130206 7803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:58.130214 7799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:58.130452 7763 server_base.cc:1034] running on GCE node
I20260810 07:22:58.130654 7763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:58.130935 7763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:58.132098 7763 hybrid_clock.cc:648] HybridClock initialized: now 1786346578132069 us; error 51 us; skew 500 ppm
Aug 10 07:22:58 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:58.134974 7763 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:22:58.135602 7763 webserver.cc:460] Webserver started at http://127.14.164.1:46495/ using document root <none> and password file <none>
I20260810 07:22:58.135897 7763 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:58.135962 7763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:58.137610 7763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:58.138417 7813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.138588 7763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:58.138713 7763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "3d9bca5957374235a6ed42d64be7617a"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "5787a387650119c4419c77be15746ded"
server_key_iv: "a22983931eee24ba6339ca29bca5e78e"
server_key_version: "encryptionkey@0"
I20260810 07:22:58.139106 7763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:58.151558 7763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:58.154183 7763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:58.154357 7763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:58.154667 7763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:58.155222 7824 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:58.157073 7763 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:58.157155 7763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:22:58.157202 7763 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:58.157935 7763 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:58.157987 7763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:22:58.158156 7824 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Bootstrap starting.
I20260810 07:22:58.167080 7763 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:34815
I20260810 07:22:58.167630 7763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:58.168206 7995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:34815 every 8 connection(s)
I20260810 07:22:58.172230 7824 log.cc:824] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:58.174919 7824 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:58.175288 7824 tablet_bootstrap.cc:492] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Bootstrap complete.
I20260810 07:22:58.176378 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 7763
I20260810 07:22:58.176668 7824 ts_tablet_manager.cc:1397] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.004s
I20260810 07:22:58.176705 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:45255
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:42923
--webserver_port=36227
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:58.177815 7824 raft_consensus.cc:348] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.178057 7824 raft_consensus.cc:729] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9bca5957374235a6ed42d64be7617a, State: Initialized, Role: FOLLOWER
I20260810 07:22:58.178210 7824 consensus_queue.cc:260] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.178534 7824 ts_tablet_manager.cc:1428] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
Aug 10 07:22:58 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:58.185457 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.168554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:37569)
0810 07:22:58.168974 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.169040 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.169870 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.170696 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.170701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.171114 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:58.171407 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.171414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.173296 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.173297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.174693 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.174696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.175675 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.175676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.175855 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.178893 (+ 3038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:58.178911 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:58.180447 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:58.183117 (+ 2670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:58.183127 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:58.183131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:58.183587 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:58.183954 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:58.183956 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:58.183956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:58.184120 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:58.184698 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:58.184707 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:58.184903 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.185024 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.185214 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":105,"threads_started":1}
I20260810 07:22:58.186159 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.168669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37569 (local address 127.14.164.62:42923)
0810 07:22:58.170141 (+ 1472us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.170153 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.170204 (+ 51us) server_negotiation.cc:407] Connection header received
0810 07:22:58.170372 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.170381 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.170444 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.170620 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:58.171571 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.173149 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.174809 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.175547 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.175669 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.180621 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:58.180627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:58.180628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:58.180644 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:58.182938 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.183702 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.183706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.183707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.183861 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.184207 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.184208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.184208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.184591 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:58.184743 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.185798 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.185956 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1402,"thread_start_us":124,"threads_started":1}
I20260810 07:22:58.186911 8000 heartbeater.cc:347] Connected to a master server at 127.14.164.62:42923
I20260810 07:22:58.187024 8000 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:58.187220 8000 heartbeater.cc:511] Master 127.14.164.62:42923 requested a full tablet report, sending...
I20260810 07:22:58.188112 7709 ts_manager.cc:194] Registered new tserver with Master: 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815)
I20260810 07:22:58.189644 7709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:37569
I20260810 07:22:58.190819 8000 heartbeater.cc:503] Master 127.14.164.62:42923 was elected leader, sending a full tablet report...
I20260810 07:22:58.197433 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.189146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.14.164.1:34815)
0810 07:22:58.193143 (+ 3997us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.193149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.193158 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:22:58.193195 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.193197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.193214 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.193327 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:58.193571 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.194614 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.196455 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.197031 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.197094 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.197182 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.197325 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.197377 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3963,"thread_start_us":77,"threads_started":1}
I20260810 07:22:58.197463 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.188877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34815 (local address 127.0.0.1:36366)
0810 07:22:58.192873 (+ 3996us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.192918 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.192998 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.193373 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.193377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.193381 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:58.193467 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.193469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.194772 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.194773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.196321 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.196324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.197112 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.197113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.197210 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.197225 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.197310 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.197372 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3956,"thread_start_us":104,"threads_started":1}
I20260810 07:22:58.197921 7887 tablet_service.cc:1514] Processing DeleteTablet for tablet 638524d9d7034e5cb79ca0ce9dd6c2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:37 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:36366
I20260810 07:22:58.198619 8012 tablet_replica.cc:330] stopping tablet replica
I20260810 07:22:58.198732 8012 raft_consensus.cc:2229] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:58.198810 8012 raft_consensus.cc:2258] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:58.199157 8012 ts_tablet_manager.cc:1905] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:58.201767 8012 ts_tablet_manager.cc:1918] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:58.201849 8012 log.cc:1196] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal/wals/638524d9d7034e5cb79ca0ce9dd6c2ac
I20260810 07:22:58.202135 8012 ts_tablet_manager.cc:1939] T 638524d9d7034e5cb79ca0ce9dd6c2ac P 3d9bca5957374235a6ed42d64be7617a: Deleting consensus metadata
I20260810 07:22:58.202723 7686 catalog_manager.cc:4841] TS 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815): tablet 638524d9d7034e5cb79ca0ce9dd6c2ac (table test-table [id=d7680d4119764744b643cffddfb8ca26]) successfully deleted
W20260810 07:22:58.273902 8008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:58.274163 8008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:58.274187 8008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:58.275403 8008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:58.275449 8008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:58.275476 8008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:22:58.277020 8008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:45255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=36227
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:58.277240 8008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:58.277438 8008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:58.283627 8026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:58.283627 8018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:58.283867 8019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:58.284650 8008 server_base.cc:1034] running on GCE node
I20260810 07:22:58.284960 8008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:58.285315 8008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:58.286499 8008 hybrid_clock.cc:648] HybridClock initialized: now 1786346578286467 us; error 46 us; skew 500 ppm
Aug 10 07:22:58 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:58.291749 8008 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:22:58.292703 8008 webserver.cc:460] Webserver started at http://127.14.164.2:36227/ using document root <none> and password file <none>
I20260810 07:22:58.293049 8008 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:58.293150 8008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:58.296255 8008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:58.297184 8037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.297371 8008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:22:58.297489 8008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "20c8afa7ed8b4918856791fb369e4456"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "3ae56a245f1462ba2735c77efa0b3591"
server_key_iv: "960b36fe1e18327629e0d67c061e1c06"
server_key_version: "encryptionkey@0"
I20260810 07:22:58.297892 8008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:58.305691 8008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:58.307770 8008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:58.307946 8008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:58.308182 8008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:58.308501 8008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:58.308555 8008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.308596 8008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:58.308624 8008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.314870 8008 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:45255
I20260810 07:22:58.314949 8227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:45255 every 8 connection(s)
I20260810 07:22:58.315266 8008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:58.317297 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 8008
I20260810 07:22:58.317606 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:40099
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:42923
--webserver_port=42015
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:22:58 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:58.327312 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.316061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:59051 (local address 127.14.164.62:42923)
0810 07:22:58.316126 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.316128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.316352 (+ 224us) server_negotiation.cc:407] Connection header received
0810 07:22:58.316975 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.316976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.316991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.317051 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:58.318165 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.319018 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.320663 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.320973 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.321040 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.323755 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:58.323756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:58.323756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:58.323801 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:58.325602 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.326175 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.326176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.326176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.326247 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.326559 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.326559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.326559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.326673 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:58.326729 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.327122 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.327237 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:22:58.327555 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.316046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:59051)
0810 07:22:58.316219 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.316257 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.316818 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.317117 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.317120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.317821 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:58.318063 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.318069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.319131 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.319132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.320556 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.320560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.321066 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.321068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.321222 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.321785 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:58.321796 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:58.323635 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:58.325701 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:58.325717 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:58.325719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:58.326081 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:58.326322 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:58.326323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:58.326323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:58.326481 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:58.326748 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:58.326756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:58.326939 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.327070 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.327265 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260810 07:22:58.328212 8228 heartbeater.cc:347] Connected to a master server at 127.14.164.62:42923
I20260810 07:22:58.328341 8228 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:58.328594 8228 heartbeater.cc:511] Master 127.14.164.62:42923 requested a full tablet report, sending...
I20260810 07:22:58.329082 7709 ts_manager.cc:194] Registered new tserver with Master: 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255)
I20260810 07:22:58.329599 7709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:59051
W20260810 07:22:58.408839 8231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:58.409050 8231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:58.409075 8231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:58.410290 8231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:58.410336 8231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:58.410362 8231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:22:58.411832 8231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:40099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=42015
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:22:58.412096 8231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:58.412365 8231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:58.418130 8231 server_base.cc:1034] running on GCE node
W20260810 07:22:58.418180 8247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:58.418104 8245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:58.418098 8244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:58.418529 8231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:58.418797 8231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:58.419977 8231 hybrid_clock.cc:648] HybridClock initialized: now 1786346578419973 us; error 38 us; skew 500 ppm
Aug 10 07:22:58 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:58.422703 8231 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:22:58.423121 8231 webserver.cc:460] Webserver started at http://127.14.164.3:42015/ using document root <none> and password file <none>
I20260810 07:22:58.423327 8231 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:58.423384 8231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:58.424862 8231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:58.425526 8259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.425761 8231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:58.425865 8231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "671dd9929f9e44d9822a1510da876fb6"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "d10d01a5511624fdfadcc3b54a8ba0ef"
server_key_iv: "9538e39faedbafc523ff8a7ddc589310"
server_key_version: "encryptionkey@0"
I20260810 07:22:58.426227 8231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:58.441890 8231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:58.444460 8231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:58.444585 8231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:58.444880 8231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:58.445188 8231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:58.445221 8231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.445245 8231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:58.445257 8231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.451186 8231 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:40099
I20260810 07:22:58.451304 8427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:40099 every 8 connection(s)
I20260810 07:22:58.451642 8231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:58.455081 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 8231
Aug 10 07:22:58 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:22:58.463611 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.455572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.14.164.62:42923)
0810 07:22:58.455692 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.455694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.455793 (+ 99us) server_negotiation.cc:407] Connection header received
0810 07:22:58.455871 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.455872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.455883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.455941 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:58.456712 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.457319 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.458485 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.458791 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.458878 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.460337 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:58.460339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:58.460339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:58.460346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:58.462275 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.462712 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.462713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.462713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.462759 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.462970 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.462970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.462971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.463082 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:58.463270 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.463400 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.463491 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260810 07:22:58.463759 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.452461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:53515 (local address 127.14.164.62:42923)
0810 07:22:58.452525 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.452527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.452825 (+ 298us) server_negotiation.cc:407] Connection header received
0810 07:22:58.453417 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.453418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.453431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.453491 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:58.454488 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.455000 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.456523 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.456794 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.456910 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.460010 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:58.460011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:58.460011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:58.460019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:58.461960 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.462536 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.462537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.462538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.462593 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.462895 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.462896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.462896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.463083 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:58.463162 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.463590 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.463660 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:22:58.464097 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.452452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:53515)
0810 07:22:58.452640 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.452678 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.453302 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.453562 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.453568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.454170 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:58.454382 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.454387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.455111 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.455112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.456417 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.456421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.456960 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.456961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.457092 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.457636 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:58.457647 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:58.459890 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:58.462060 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:58.462067 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:58.462069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:58.462462 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:58.462664 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:58.462665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:58.462665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:58.462820 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:58.463218 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:58.463227 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:58.463408 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.463559 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.463752 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260810 07:22:58.464861 8429 heartbeater.cc:347] Connected to a master server at 127.14.164.62:42923
I20260810 07:22:58.464968 8429 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:58.465129 8429 heartbeater.cc:511] Master 127.14.164.62:42923 requested a full tablet report, sending...
I20260810 07:22:58.465504 7708 ts_manager.cc:194] Registered new tserver with Master: 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
I20260810 07:22:58.466049 7708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:53515
I20260810 07:22:58.475844 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:58.479354 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.476415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.14.164.62:42923)
0810 07:22:58.476491 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.476493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.476644 (+ 151us) server_negotiation.cc:407] Connection header received
0810 07:22:58.476761 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.476763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.476825 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.476920 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:58.477086 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.477612 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.478473 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.478686 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.478771 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.479010 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:58.479076 (+ 66us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 07:22:58.479250 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59764: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 07:22:58.479455 8004 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59764: token expired
W20260810 07:22:58.479632 7595 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.14.164.62:42923)
I20260810 07:22:58.486599 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.479990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.14.164.62:42923)
0810 07:22:58.480044 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.480046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.480057 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:58.480083 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.480084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.480093 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.480144 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:58.480707 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.481209 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.481968 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.482173 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.482244 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.483570 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:58.483572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:58.483572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:58.483580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:58.485200 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.485621 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.485622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.485623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.485693 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:58.485963 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:58.485964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:58.485965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:58.486150 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:58.486265 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.486389 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.486485 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:22:58.488220 7708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:58.490049 7708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:58.492547 7887 tablet_service.cc:1467] Processing CreateTablet for tablet 466bf0126e734ea69b187dae8ec63c50 (DEFAULT_TABLE table=test-table [id=6e240074fb0e4b04b85b7d5c4e7198ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:58.492820 7887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 466bf0126e734ea69b187dae8ec63c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:58.494539 7824 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Bootstrap starting.
I20260810 07:22:58.494976 7824 tablet_bootstrap.cc:654] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:58.495793 7824 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: No bootstrap required, opened a new log
I20260810 07:22:58.495850 7824 ts_tablet_manager.cc:1397] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:58.495968 7824 raft_consensus.cc:348] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.496039 7824 raft_consensus.cc:374] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:58.496091 7824 raft_consensus.cc:729] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9bca5957374235a6ed42d64be7617a, State: Initialized, Role: FOLLOWER
I20260810 07:22:58.496104 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.492417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.14.164.2:45255)
0810 07:22:58.492620 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.492626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.492639 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:22:58.492687 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.492689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.492706 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.492814 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:58.492943 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.493839 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.494802 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.495781 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.495846 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.495913 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.495980 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.496021 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260810 07:22:58.496095 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.492121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:45255 (local address 127.0.0.1:48094)
0810 07:22:58.492217 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.492232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.492281 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.492822 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.492823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.492825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:58.492880 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.492882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.493909 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.493910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.494722 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.494726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.495843 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.495844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.495909 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.495913 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.495967 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.496021 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 07:22:58.496152 7824 consensus_queue.cc:260] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.496238 7824 ts_tablet_manager.cc:1428] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:58.497272 8122 tablet_service.cc:1467] Processing CreateTablet for tablet 466bf0126e734ea69b187dae8ec63c50 (DEFAULT_TABLE table=test-table [id=6e240074fb0e4b04b85b7d5c4e7198ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:58.497617 8122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 466bf0126e734ea69b187dae8ec63c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:58.498939 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.492232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:40099 (local address 127.0.0.1:52022)
0810 07:22:58.494605 (+ 2373us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.494618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.494685 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.494962 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.494962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.494965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:58.495010 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.495011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.496502 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.496503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.497627 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.497630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.498700 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.498702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.498757 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.498761 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.498812 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.498879 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2358,"thread_start_us":89,"threads_started":1}
I20260810 07:22:58.498974 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.492418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.14.164.3:40099)
0810 07:22:58.492677 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.492681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.494764 (+ 2083us) server_negotiation.cc:407] Connection header received
0810 07:22:58.494822 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.494825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.494845 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.494950 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:58.495093 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.496336 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.497720 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.498593 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.498683 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.498745 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.498839 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.498889 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":135,"threads_started":1}
I20260810 07:22:58.499644 8333 tablet_service.cc:1467] Processing CreateTablet for tablet 466bf0126e734ea69b187dae8ec63c50 (DEFAULT_TABLE table=test-table [id=6e240074fb0e4b04b85b7d5c4e7198ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:58.499938 8333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 466bf0126e734ea69b187dae8ec63c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:58.501520 8443 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Bootstrap starting.
I20260810 07:22:58.502192 8443 tablet_bootstrap.cc:654] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:58.502717 8443 log.cc:824] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:58.503073 8445 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Bootstrap starting.
I20260810 07:22:58.503532 8443 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: No bootstrap required, opened a new log
I20260810 07:22:58.503611 8443 ts_tablet_manager.cc:1397] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:58.503916 8445 tablet_bootstrap.cc:654] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:58.504426 8445 log.cc:824] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:58.505249 8445 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: No bootstrap required, opened a new log
I20260810 07:22:58.505340 8445 ts_tablet_manager.cc:1397] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:58.505415 8443 raft_consensus.cc:348] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.505602 8443 raft_consensus.cc:374] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:58.505627 8443 raft_consensus.cc:729] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c8afa7ed8b4918856791fb369e4456, State: Initialized, Role: FOLLOWER
I20260810 07:22:58.505780 8443 consensus_queue.cc:260] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.506078 8443 ts_tablet_manager.cc:1428] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:22:58.506109 8228 heartbeater.cc:503] Master 127.14.164.62:42923 was elected leader, sending a full tablet report...
I20260810 07:22:58.507423 8445 raft_consensus.cc:348] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.507583 8445 raft_consensus.cc:374] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:58.507630 8445 raft_consensus.cc:729] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671dd9929f9e44d9822a1510da876fb6, State: Initialized, Role: FOLLOWER
I20260810 07:22:58.507747 8445 consensus_queue.cc:260] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.508080 8445 ts_tablet_manager.cc:1428] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:22:58.508076 8429 heartbeater.cc:503] Master 127.14.164.62:42923 was elected leader, sending a full tablet report...
I20260810 07:22:58.516165 8450 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:58.516284 8450 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.516625 8450 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815)
I20260810 07:22:58.520562 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.516905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:46011 (local address 127.14.164.2:45255)
0810 07:22:58.516972 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.516975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.516986 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:58.517019 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.517020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.517034 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.517099 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:58.517287 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.518144 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.519642 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.520178 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.520267 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.520277 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.520446 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.520487 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:22:58.520602 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.516807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:45255 (local address 127.14.164.3:46011)
0810 07:22:58.516865 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.516878 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.516922 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.517122 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.517123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.517126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:58.517207 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.517208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.518243 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.518244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.519532 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.519535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.520275 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.520276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.520371 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.520374 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.520439 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.520486 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":11}
I20260810 07:22:58.520628 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.516951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:42351 (local address 127.14.164.1:34815)
0810 07:22:58.516995 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.516997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.517168 (+ 171us) server_negotiation.cc:407] Connection header received
0810 07:22:58.517209 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.517211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.517219 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.517300 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:58.517452 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.518261 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.519689 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.520229 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.520331 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.520340 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.520500 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.520548 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:22:58.520695 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.516884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34815 (local address 127.14.164.3:42351)
0810 07:22:58.517087 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.517097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:58.517150 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:58.517311 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:58.517311 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:58.517313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:58.517384 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.517385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.518372 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.518373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.519532 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:58.519535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.520322 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.520323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:58.520413 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.520417 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:22:58.520493 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:22:58.520540 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":10,"thread_start_us":132,"threads_started":1}
I20260810 07:22:58.521008 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466bf0126e734ea69b187dae8ec63c50" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8afa7ed8b4918856791fb369e4456" is_pre_election: true
I20260810 07:22:58.521201 8155 raft_consensus.cc:2454] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 0.
I20260810 07:22:58.521241 7930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466bf0126e734ea69b187dae8ec63c50" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9bca5957374235a6ed42d64be7617a" is_pre_election: true
I20260810 07:22:58.521426 7930 raft_consensus.cc:2454] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 0.
I20260810 07:22:58.521456 8264 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6; no voters:
I20260810 07:22:58.521615 8450 raft_consensus.cc:2781] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:58.521692 8450 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:58.521729 8450 raft_consensus.cc:3037] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:58.522516 8450 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.522673 8450 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 election: Requested vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815)
I20260810 07:22:58.522840 7930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466bf0126e734ea69b187dae8ec63c50" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9bca5957374235a6ed42d64be7617a"
I20260810 07:22:58.522866 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466bf0126e734ea69b187dae8ec63c50" candidate_uuid: "671dd9929f9e44d9822a1510da876fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8afa7ed8b4918856791fb369e4456"
I20260810 07:22:58.522956 7930 raft_consensus.cc:3037] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:58.522951 8155 raft_consensus.cc:3037] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:58.523818 8155 raft_consensus.cc:2454] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 1.
I20260810 07:22:58.524003 8264 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6; no voters:
I20260810 07:22:58.524169 7930 raft_consensus.cc:2454] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671dd9929f9e44d9822a1510da876fb6 in term 1.
I20260810 07:22:58.524303 8450 raft_consensus.cc:2781] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:58.524590 8450 raft_consensus.cc:686] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 671dd9929f9e44d9822a1510da876fb6, State: Running, Role: LEADER
I20260810 07:22:58.524721 8450 consensus_queue.cc:237] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:22:58.525861 7709 catalog_manager.cc:5495] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3). New cstate: current_term: 1 leader_uuid: "671dd9929f9e44d9822a1510da876fb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } health_report { overall_health: UNKNOWN } } }
I20260810 07:22:58.539422 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.536704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52038 (local address 127.14.164.3:40099)
0810 07:22:58.536837 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:58.536839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:58.536849 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:22:58.536886 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:58.536887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:58.536906 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:58.536977 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:58.537098 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.537625 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.538466 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:58.538671 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:58.538740 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:58.538966 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:58.539090 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:58.539201 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:58.539260 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 07:22:58.539284 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:22:58.541692 7930 raft_consensus.cc:1261] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Refusing update from remote peer 671dd9929f9e44d9822a1510da876fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:58.541692 8155 raft_consensus.cc:1261] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Refusing update from remote peer 671dd9929f9e44d9822a1510da876fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:58.542143 8453 consensus_queue.cc:1035] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:58.542284 8450 consensus_queue.cc:1035] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:58.544766 8457 mvcc.cc:205] Tried to move back new op lower bound from 7316875585703731200 to 7316875585638494208. Current Snapshot: MvccSnapshot[applied={T|T < 7316875585703731200}]
I20260810 07:22:58.546968 7708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:58.547088 7708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:58.547817 7708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Sending DeleteTablet for 3 replicas of tablet 466bf0126e734ea69b187dae8ec63c50
I20260810 07:22:58.548118 7887 tablet_service.cc:1514] Processing DeleteTablet for tablet 466bf0126e734ea69b187dae8ec63c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:58 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:36366
I20260810 07:22:58.548226 8333 tablet_service.cc:1514] Processing DeleteTablet for tablet 466bf0126e734ea69b187dae8ec63c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:58 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:52022
I20260810 07:22:58.548246 8122 tablet_service.cc:1514] Processing DeleteTablet for tablet 466bf0126e734ea69b187dae8ec63c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:58 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:48094
I20260810 07:22:58.548285 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 7763
I20260810 07:22:58.548449 8012 tablet_replica.cc:330] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: stopping tablet replica
I20260810 07:22:58.548507 8012 raft_consensus.cc:2229] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:58.548559 8012 raft_consensus.cc:2258] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:58.553987 8472 tablet_replica.cc:330] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: stopping tablet replica
I20260810 07:22:58.554169 8472 raft_consensus.cc:2229] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:22:58.554354 8472 raft_consensus.cc:2258] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:58.555038 8470 tablet_replica.cc:330] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: stopping tablet replica
I20260810 07:22:58.555187 8470 raft_consensus.cc:2229] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:58.555325 8470 raft_consensus.cc:2258] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:58.556203 8472 ts_tablet_manager.cc:1905] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:58.559399 8470 ts_tablet_manager.cc:1905] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:58.562310 8470 ts_tablet_manager.cc:1918] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:58.562562 8470 log.cc:1196] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/466bf0126e734ea69b187dae8ec63c50
I20260810 07:22:58.563090 8470 ts_tablet_manager.cc:1939] T 466bf0126e734ea69b187dae8ec63c50 P 20c8afa7ed8b4918856791fb369e4456: Deleting consensus metadata
I20260810 07:22:58.563830 7683 catalog_manager.cc:4841] TS 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): tablet 466bf0126e734ea69b187dae8ec63c50 (table test-table [id=6e240074fb0e4b04b85b7d5c4e7198ce]) successfully deleted
I20260810 07:22:58.565232 8472 ts_tablet_manager.cc:1918] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:58.565326 8472 log.cc:1196] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/466bf0126e734ea69b187dae8ec63c50
I20260810 07:22:58.565645 8472 ts_tablet_manager.cc:1939] T 466bf0126e734ea69b187dae8ec63c50 P 671dd9929f9e44d9822a1510da876fb6: Deleting consensus metadata
I20260810 07:22:58.566131 7683 catalog_manager.cc:4841] TS 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): tablet 466bf0126e734ea69b187dae8ec63c50 (table test-table [id=6e240074fb0e4b04b85b7d5c4e7198ce]) successfully deleted
W20260810 07:22:58.572202 7686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:34815 (error 108)
I20260810 07:22:58.572690 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 8008
I20260810 07:22:58.572887 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.572686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:34815 (local address 127.0.0.1:36372)
0810 07:22:58.572782 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.572842 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 07:22:58.584044 7686 catalog_manager.cc:4584] TS 3d9bca5957374235a6ed42d64be7617a (127.14.164.1:34815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 466bf0126e734ea69b187dae8ec63c50: Network error: Client connection negotiation failed: client connection to 127.14.164.1:34815: connect: Connection refused (error 111)
I20260810 07:22:58.588331 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 8231
I20260810 07:22:58.601083 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 7655
I20260810 07:23:18.613343 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42923
--webserver_interface=127.14.164.62
--webserver_port=42481
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:18.699918 8474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:18.700218 8474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:18.700268 8474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:18.701651 8474 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:23:18.701735 8474 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:23:18.701763 8474 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:23:18.701799 8474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:18.701826 8474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:18.701859 8474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:18.701893 8474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:18.701922 8474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:18.703653 8474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=42481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:18.703986 8474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:18.704270 8474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:18.709825 8482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:18.709965 8474 server_base.cc:1034] running on GCE node
W20260810 07:23:18.710060 8483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:18.709839 8485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:18.710438 8474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:18.710762 8474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:18.711951 8474 hybrid_clock.cc:648] HybridClock initialized: now 1786346598711925 us; error 40 us; skew 500 ppm
Aug 10 07:23:18 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346598, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:18.718700 8474 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:23:18.719316 8474 webserver.cc:460] Webserver started at http://127.14.164.62:42481/ using document root <none> and password file <none>
I20260810 07:23:18.719587 8474 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:18.719659 8474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:18.721247 8474 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:23:18.722137 8495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:18.722393 8474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:18.722519 8474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "efd43fc9ea2c42d6a200c41ae4143ed1"
format_stamp: "Formatted at 2026-08-10 07:22:36 on dist-test-slave-x5fp"
server_key: "98f84e9f44610069690e2d057e47c77d"
server_key_iv: "32ab84e83e9c1a883bb3d78d04d8ffc2"
server_key_version: "encryptionkey@0"
I20260810 07:23:18.722939 8474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:18.735321 8474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:18.738075 8474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:18.738273 8474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:18.742902 8474 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42923
I20260810 07:23:18.742967 8577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42923 every 8 connection(s)
I20260810 07:23:18.743418 8474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:18.745190 8579 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:23:18.745992 8579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap starting.
I20260810 07:23:18.750005 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 8474
I20260810 07:23:18.750316 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:34815
--local_ip_for_outbound_sockets=127.14.164.1
--tserver_master_addrs=127.14.164.62:42923
--webserver_port=46495
--webserver_interface=127.14.164.1
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:23:18.756943 8579 log.cc:824] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:18.761121 8579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:23:18.761562 8579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Bootstrap complete.
I20260810 07:23:18.765595 8579 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:23:18.765851 8579 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd43fc9ea2c42d6a200c41ae4143ed1, State: Initialized, Role: FOLLOWER
I20260810 07:23:18.766024 8579 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:23:18.766095 8579 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:18.766163 8579 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:18.766207 8579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:23:18.767347 8579 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:23:18.767472 8579 leader_election.cc:302] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd43fc9ea2c42d6a200c41ae4143ed1; no voters:
I20260810 07:23:18.767678 8579 leader_election.cc:288] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:23:18.767724 8585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:23:18.767863 8585 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [term 3 LEADER]: Becoming Leader. State: Replica: efd43fc9ea2c42d6a200c41ae4143ed1, State: Running, Role: LEADER
I20260810 07:23:18.767970 8579 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:18.767956 8585 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } }
I20260810 07:23:18.768257 8585 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd43fc9ea2c42d6a200c41ae4143ed1. Latest consensus state: current_term: 3 leader_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } } }
I20260810 07:23:18.768344 8585 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:18.768262 8588 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd43fc9ea2c42d6a200c41ae4143ed1" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42923 } } }
I20260810 07:23:18.768555 8588 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:18.768700 8593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:18.769650 8593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:18.769860 8593 catalog_manager.cc:1259] Loaded cluster ID: 86243680af6e441d84ddecebb7aafa39
I20260810 07:23:18.770022 8593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:18.770859 8593 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:18.771051 8593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Loaded TSK: 0
I20260810 07:23:18.771107 8593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P efd43fc9ea2c42d6a200c41ae4143ed1: Loaded TSK: 1
F20260810 07:23:18.771339 8593 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f13ed3119ad google::LogMessage::Fail() at ??:0
@ 0x7f13ed31488a google::LogMessage::SendToLog() at ??:0
@ 0x7f13ed3114de google::LogMessage::Flush() at ??:0
@ 0x7f13ed312a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f13efa7e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f13efa7bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f13f526b1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f13f526b48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f13f526b594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f13f5231c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f13f523bad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f13ef1df0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f13ef1d4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f13f00386db start_thread at ??:0
@ 0x7f13ebe0f61f clone at ??:0
W20260810 07:23:18.862571 8583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:18.862838 8583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:18.862881 8583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:18.864076 8583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:18.864144 8583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:18.864188 8583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:18.865980 8583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:34815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=46495
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:18.866305 8583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:18.866595 8583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:18.873133 8649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:18.873135 8650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:18.873333 8583 server_base.cc:1034] running on GCE node
W20260810 07:23:18.873343 8653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:18.873752 8583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:18.874078 8583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:18.875299 8583 hybrid_clock.cc:648] HybridClock initialized: now 1786346598875278 us; error 33 us; skew 500 ppm
Aug 10 07:23:18 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346598, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:18.878545 8583 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:23:18.879127 8583 webserver.cc:460] Webserver started at http://127.14.164.1:46495/ using document root <none> and password file <none>
I20260810 07:23:18.879427 8583 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:18.879510 8583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:18.881127 8583 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:23:18.881980 8663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:18.882318 8583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:18.882436 8583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "3d9bca5957374235a6ed42d64be7617a"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "5787a387650119c4419c77be15746ded"
server_key_iv: "a22983931eee24ba6339ca29bca5e78e"
server_key_version: "encryptionkey@0"
I20260810 07:23:18.882993 8583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:18.934800 8583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:18.937681 8583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:18.937855 8583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:18.938162 8583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:18.938731 8672 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:23:18.940637 8583 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:23:18.940740 8583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:23:18.940817 8583 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:23:18.941792 8583 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:23:18.941843 8583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:18.941970 8672 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Bootstrap starting.
I20260810 07:23:18.949640 8583 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:34815
I20260810 07:23:18.950150 8583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:18.950706 8840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:34815 every 8 connection(s)
I20260810 07:23:18.953814 8672 log.cc:824] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:18.956312 8672 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:23:18.956627 8672 tablet_bootstrap.cc:492] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Bootstrap complete.
I20260810 07:23:18.957711 8672 ts_tablet_manager.cc:1397] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260810 07:23:18.958814 8672 raft_consensus.cc:348] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:18.959084 8672 raft_consensus.cc:729] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9bca5957374235a6ed42d64be7617a, State: Initialized, Role: FOLLOWER
I20260810 07:23:18.959280 8672 consensus_queue.cc:260] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:18.959470 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 8583
I20260810 07:23:18.959618 8672 ts_tablet_manager.cc:1428] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 07:23:18.959774 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:45255
--local_ip_for_outbound_sockets=127.14.164.2
--tserver_master_addrs=127.14.164.62:42923
--webserver_port=36227
--webserver_interface=127.14.164.2
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:19.030471 8844 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:23:18.950947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:57657)
0810 07:23:18.951209 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:18.951268 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:18.951906 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:19.030209 (+ 78303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":132,"threads_started":1}
W20260810 07:23:19.030793 8841 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:42923 (0 consecutive failures): Network error: Failed to ping master at 127.14.164.62:42923: Client connection negotiation failed: client connection to 127.14.164.62:42923: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:23:19.031188 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.031054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:56993)
0810 07:23:19.031114 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.031157 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:23:19.031569 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.031480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:35873)
0810 07:23:19.031527 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.031552 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:23:19.031701 8841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:23:19.067726 8848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:19.068017 8848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:19.068063 8848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:19.069425 8848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:19.069506 8848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:19.069552 8848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:19.071125 8848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:45255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=36227
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:19.071439 8848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:19.071702 8848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:19.077419 8854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:19.077409 8855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:19.077409 8857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:19.077736 8848 server_base.cc:1034] running on GCE node
I20260810 07:23:19.077953 8848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:19.078181 8848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:19.079371 8848 hybrid_clock.cc:648] HybridClock initialized: now 1786346599079350 us; error 52 us; skew 500 ppm
Aug 10 07:23:19 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346599, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:19.082243 8848 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:23:19.082705 8848 webserver.cc:460] Webserver started at http://127.14.164.2:36227/ using document root <none> and password file <none>
I20260810 07:23:19.082962 8848 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:19.083014 8848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:19.084542 8848 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:23:19.085372 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:19.085628 8848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:19.085739 8848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "20c8afa7ed8b4918856791fb369e4456"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "3ae56a245f1462ba2735c77efa0b3591"
server_key_iv: "960b36fe1e18327629e0d67c061e1c06"
server_key_version: "encryptionkey@0"
I20260810 07:23:19.086154 8848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:19.098002 8848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:19.100921 8848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:19.101116 8848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:19.101426 8848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:19.101804 8848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:19.101864 8848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:19.101905 8848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:19.101959 8848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:19.108955 8848 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:45255
I20260810 07:23:19.109042 9045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:45255 every 8 connection(s)
I20260810 07:23:19.109460 8848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:19.110666 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.110217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:45329)
0810 07:23:19.110422 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.110504 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
W20260810 07:23:19.111140 9046 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:42923 (0 consecutive failures): Network error: Failed to ping master at 127.14.164.62:42923: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
I20260810 07:23:19.111537 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.111392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:43739)
0810 07:23:19.111478 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.111513 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 07:23:19.111932 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.111832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:35451)
0810 07:23:19.111893 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.111916 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260810 07:23:19.112121 9046 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:23:19.117584 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 8848
I20260810 07:23:19.117921 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:40099
--local_ip_for_outbound_sockets=127.14.164.3
--tserver_master_addrs=127.14.164.62:42923
--webserver_port=42015
--webserver_interface=127.14.164.3
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:19.205446 9049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:19.205677 9049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:19.205700 9049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:19.206845 9049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:19.206881 9049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:19.206908 9049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:19.208328 9049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:40099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=42015
--tserver_master_addrs=127.14.164.62:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:19.208539 9049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:19.208793 9049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:19.214146 9061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:19.214170 9057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:19.214078 9058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:19.214337 9049 server_base.cc:1034] running on GCE node
I20260810 07:23:19.214540 9049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:19.214752 9049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:19.215921 9049 hybrid_clock.cc:648] HybridClock initialized: now 1786346599215875 us; error 55 us; skew 500 ppm
Aug 10 07:23:19 dist-test-slave-x5fp krb5kdc[6790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346599, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:19.218662 9049 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:23:19.219087 9049 webserver.cc:460] Webserver started at http://127.14.164.3:42015/ using document root <none> and password file <none>
I20260810 07:23:19.219300 9049 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:19.219342 9049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:19.220886 9049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:19.221490 9072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:19.221698 9049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:19.221798 9049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "671dd9929f9e44d9822a1510da876fb6"
format_stamp: "Formatted at 2026-08-10 07:22:37 on dist-test-slave-x5fp"
server_key: "d10d01a5511624fdfadcc3b54a8ba0ef"
server_key_iv: "9538e39faedbafc523ff8a7ddc589310"
server_key_version: "encryptionkey@0"
I20260810 07:23:19.222148 9049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:19.229476 9049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:19.231815 9049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:19.231935 9049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:19.232125 9049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:19.232496 9049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:19.232533 9049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:19.232558 9049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:19.232606 9049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:19.239097 9049 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:40099
I20260810 07:23:19.239259 9249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:40099 every 8 connection(s)
I20260810 07:23:19.239513 9049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:19.240623 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.240205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:50133)
0810 07:23:19.240371 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.240468 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
W20260810 07:23:19.241046 9252 heartbeater.cc:644] Failed to heartbeat to 127.14.164.62:42923 (0 consecutive failures): Network error: Failed to ping master at 127.14.164.62:42923: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
I20260810 07:23:19.241353 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.241250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:58205)
0810 07:23:19.241301 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.241332 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 07:23:19.241704 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:19.241612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:59267)
0810 07:23:19.241656 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:19.241680 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:23:19.241806 9252 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:23:19.244817 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 9049
I20260810 07:23:20.032480 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.032174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:44303)
0810 07:23:20.032348 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:20.032396 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260810 07:23:20.112901 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.112524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:35231)
0810 07:23:20.112708 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:20.112838 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260810 07:23:20.242434 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.242185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:46099)
0810 07:23:20.242334 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:20.242377 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260810 07:23:20.531319 9263 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:20.531481 9263 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:20.532003 9263 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
Aug 10 07:23:20 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346598, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.2@KRBTEST.COM
Aug 10 07:23:20 dist-test-slave-x5fp krb5kdc[6790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346598, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.3@KRBTEST.COM
I20260810 07:23:20.543365 9265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.532397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:47441 (local address 127.14.164.2:45255)
0810 07:23:20.532592 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:20.532609 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:20.532641 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:23:20.533153 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:20.533155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:20.533199 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:20.533363 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:20.534363 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.535259 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.536296 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.536775 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.536883 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:20.539359 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:20.539364 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:20.539365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:20.539378 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:20.541349 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:20.541870 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:20.541874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:20.541875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:20.542000 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:20.542389 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:20.542390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:20.542390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:20.542688 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:20.542744 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:20.543106 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 07:23:20.543238 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260810 07:23:20.543347 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.532487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:40099 (local address 127.14.164.1:37455)
0810 07:23:20.532637 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:20.532651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:20.532718 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:20.533615 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:20.533616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:20.534003 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:20.534236 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:20.534244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.535455 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.535456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:20.536200 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:20.536203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.537001 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.537002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:20.537069 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:20.537763 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:20.537769 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:20.539481 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:20.541696 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:20.541697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:20.541697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:20.541927 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:20.542333 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:20.542333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:20.542334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:20.542422 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:20.542857 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:20.542857 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:20.542934 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 07:23:20.543048 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 07:23:20.543252 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":47,"thread_start_us":87,"threads_started":1}
I20260810 07:23:20.543479 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.532269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:45255 (local address 127.14.164.1:47441)
0810 07:23:20.532353 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:20.532371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:20.532469 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:20.533393 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:20.533396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:20.534007 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:20.534236 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:20.534243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.535380 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.535381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:20.536196 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:20.536198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.536876 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.536877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:20.536994 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:20.537560 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:20.537569 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:20.539187 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:20.541444 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:20.541453 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:20.541455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:20.541784 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:20.542118 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:20.542119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:20.542119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:20.542316 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:20.542782 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:20.542789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:20.542942 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 07:23:20.543048 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 07:23:20.543358 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":147}
I20260810 07:23:20.543390 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:20.532627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:37455 (local address 127.14.164.3:40099)
0810 07:23:20.532889 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:20.532907 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:20.532937 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:23:20.533405 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:20.533407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:20.533447 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:20.533649 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:20.534385 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.535331 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.536342 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:20.536882 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:20.537047 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:20.539605 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:20.539609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:20.539609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:20.539620 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:20.541601 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:20.542102 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:20.542105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:20.542106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:20.542252 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:20.542494 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:20.542494 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:20.542495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:20.542772 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:20.542864 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:20.543108 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 07:23:20.543246 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":146,"threads_started":1}
W20260810 07:23:20.544622 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
W20260810 07:23:20.544766 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
I20260810 07:23:20.544808 8667 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a; no voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6
I20260810 07:23:20.544947 9263 raft_consensus.cc:2726] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:21.033393 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:21.033202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:49785)
0810 07:23:21.033294 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:21.033339 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 07:23:21.113720 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:21.113475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:46993)
0810 07:23:21.113622 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:21.113668 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260810 07:23:21.243192 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:21.242950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:39635)
0810 07:23:21.243103 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:21.243148 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260810 07:23:22.034309 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:22.033997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:38455)
0810 07:23:22.034172 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:22.034254 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":72,"threads_started":1}
I20260810 07:23:22.114651 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:22.114395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:44697)
0810 07:23:22.114542 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:22.114599 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260810 07:23:22.243966 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:22.243733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:58439)
0810 07:23:22.243881 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:22.243920 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260810 07:23:22.781649 9279 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:22.781864 9279 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:22.782106 9279 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
W20260810 07:23:22.782662 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
W20260810 07:23:22.782745 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
I20260810 07:23:22.782766 8667 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a; no voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6
I20260810 07:23:22.782847 9279 raft_consensus.cc:2726] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:23.035145 9280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:23.034889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:56979)
0810 07:23:23.035042 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:23.035091 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260810 07:23:23.115600 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:23.115261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:46969)
0810 07:23:23.115474 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:23.115525 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
I20260810 07:23:23.244765 9282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:23.244444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:58139)
0810 07:23:23.244597 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:23.244644 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260810 07:23:24.036101 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:24.035790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:39523)
0810 07:23:24.035984 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:24.036043 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
I20260810 07:23:24.116508 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:24.116261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:49985)
0810 07:23:24.116412 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:24.116456 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260810 07:23:24.245590 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:24.245346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:52113)
0810 07:23:24.245492 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:24.245537 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260810 07:23:25.036933 9290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.036635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:47671)
0810 07:23:25.036831 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:25.036878 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260810 07:23:25.117347 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.117050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:53705)
0810 07:23:25.117240 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:25.117290 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":66,"threads_started":1}
I20260810 07:23:25.246610 9293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.246318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:45675)
0810 07:23:25.246502 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:25.246553 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":80,"threads_started":1}
I20260810 07:23:26.037691 9294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.037461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:45223)
0810 07:23:26.037596 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.037638 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260810 07:23:26.078181 9295 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:26.078310 9295 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:26.078527 9295 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
W20260810 07:23:26.079015 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
W20260810 07:23:26.079114 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
I20260810 07:23:26.079154 8667 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a; no voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6
I20260810 07:23:26.079254 9295 raft_consensus.cc:2726] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:26.118129 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.117896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:55919)
0810 07:23:26.118046 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.118079 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260810 07:23:26.247427 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.247190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:38343)
0810 07:23:26.247327 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.247374 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260810 07:23:27.038596 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:27.038307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:40555)
0810 07:23:27.038479 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:27.038541 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260810 07:23:27.119078 9303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:27.118732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:40059)
0810 07:23:27.118951 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:27.119013 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
I20260810 07:23:27.248288 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:27.247960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:37959)
0810 07:23:27.248184 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:27.248234 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
I20260810 07:23:28.039624 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:28.039282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:39061)
0810 07:23:28.039501 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:28.039555 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":105,"threads_started":1}
I20260810 07:23:28.120042 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:28.119757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:36671)
0810 07:23:28.119944 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:28.119989 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260810 07:23:28.249226 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:28.249000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:35251)
0810 07:23:28.249130 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:28.249175 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260810 07:23:28.705204 9309 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:28.705322 9309 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:28.705470 9309 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
W20260810 07:23:28.705974 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
W20260810 07:23:28.706084 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
I20260810 07:23:28.706117 8667 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a; no voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6
I20260810 07:23:28.706223 9309 raft_consensus.cc:2726] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:29.040597 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:29.040303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:37355)
0810 07:23:29.040451 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:29.040503 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260810 07:23:29.120908 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:29.120577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:57455)
0810 07:23:29.120772 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:29.120854 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260810 07:23:29.250032 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:29.249790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:52389)
0810 07:23:29.249940 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:29.249983 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260810 07:23:30.041388 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:30.041148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:39953)
0810 07:23:30.041289 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:30.041333 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260810 07:23:30.121771 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:30.121506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:34957)
0810 07:23:30.121668 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:30.121712 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260810 07:23:30.250875 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:30.250614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:57625)
0810 07:23:30.250775 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:30.250823 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260810 07:23:30.616586 9320 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:30.616698 9320 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:30.617019 9320 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
W20260810 07:23:30.617543 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
W20260810 07:23:30.617638 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
I20260810 07:23:30.617686 8667 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a; no voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6
I20260810 07:23:30.617802 9320 raft_consensus.cc:2726] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:31.042302 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:31.042042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:51925)
0810 07:23:31.042206 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:31.042247 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260810 07:23:31.122648 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:31.122386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:44979)
0810 07:23:31.122546 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:31.122593 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260810 07:23:31.251719 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:31.251475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:41655)
0810 07:23:31.251614 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:31.251663 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260810 07:23:32.043123 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:32.042866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:44615)
0810 07:23:32.043031 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:32.043072 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260810 07:23:32.123589 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:32.123314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:40283)
0810 07:23:32.123484 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:32.123532 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260810 07:23:32.252666 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:32.252336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:59861)
0810 07:23:32.252547 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:32.252599 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":130,"threads_started":1}
I20260810 07:23:32.508991 9330 raft_consensus.cc:482] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:32.509128 9330 raft_consensus.cc:504] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8afa7ed8b4918856791fb369e4456" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 45255 } } peers { permanent_uuid: "671dd9929f9e44d9822a1510da876fb6" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 40099 } } peers { permanent_uuid: "3d9bca5957374235a6ed42d64be7617a" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 34815 } }
I20260810 07:23:32.509343 9330 leader_election.cc:288] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255), 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099)
W20260810 07:23:32.509941 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671dd9929f9e44d9822a1510da876fb6 (127.14.164.3:40099): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
W20260810 07:23:32.510015 8667 leader_election.cc:341] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8afa7ed8b4918856791fb369e4456 (127.14.164.2:45255): Not found: Tablet not found: 466bf0126e734ea69b187dae8ec63c50
I20260810 07:23:32.510030 8667 leader_election.cc:302] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9bca5957374235a6ed42d64be7617a; no voters: 20c8afa7ed8b4918856791fb369e4456, 671dd9929f9e44d9822a1510da876fb6
I20260810 07:23:32.510141 9330 raft_consensus.cc:2726] T 466bf0126e734ea69b187dae8ec63c50 P 3d9bca5957374235a6ed42d64be7617a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:33.044104 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:33.043747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:33333)
0810 07:23:33.043965 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:33.044013 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":154,"threads_started":1}
I20260810 07:23:33.124472 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:33.124218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:56061)
0810 07:23:33.124383 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:33.124416 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260810 07:23:33.253544 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:33.253265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:41593)
0810 07:23:33.253434 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:33.253481 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260810 07:23:34.045086 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:34.044825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.1:54395)
0810 07:23:34.044982 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:34.045031 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260810 07:23:34.125454 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:34.125195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.2:51113)
0810 07:23:34.125324 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:34.125401 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260810 07:23:34.250342 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 8583
I20260810 07:23:34.259400 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:34.258331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:42923 (local address 127.14.164.3:50501)
0810 07:23:34.259282 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:34.259331 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.62:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":640,"threads_started":1}
I20260810 07:23:34.263895 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 8848
I20260810 07:23:34.277184 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 9049
2026-08-10T07:23:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59035 ms)
[----------] 30 tests from SecurityITest (250542 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:34 dist-test-slave-x5fp krb5kdc[9346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:34 dist-test-slave-x5fp krb5kdc[9346](info): set up 2 sockets
Aug 10 07:23:34 dist-test-slave-x5fp krb5kdc[9346](info): commencing operation
krb5kdc: starting...
W20260810 07:23:35.545192 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:35 dist-test-slave-x5fp krb5kdc[9346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:23:35.680799 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:45041
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:45041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:35.765695 9363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:35.765974 9363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:35.766016 9363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:35.767171 9363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:35.767241 9363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:35.767266 9363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:35.767295 9363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:35.767324 9363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:35.768848 9363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:45041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:35.769145 9363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:35.769388 9363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:35.774580 9375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:35.774643 9373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:35.774871 9363 server_base.cc:1034] running on GCE node
W20260810 07:23:35.774587 9372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:35.775187 9363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:35.775499 9363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:35.776676 9363 hybrid_clock.cc:648] HybridClock initialized: now 1786346615776670 us; error 37 us; skew 500 ppm
Aug 10 07:23:35 dist-test-slave-x5fp krb5kdc[9346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346615, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:35.779258 9363 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:23:35.779776 9363 webserver.cc:460] Webserver started at http://127.14.164.62:34665/ using document root <none> and password file <none>
I20260810 07:23:35.780010 9363 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:35.780081 9363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:35.780238 9363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:35.781430 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b"
format_stamp: "Formatted at 2026-08-10 07:23:35 on dist-test-slave-x5fp"
server_key: "3047e4f0508a869e1895c66808d0db68"
server_key_iv: "2a49199fa93e778f7f6c57cbb1d8e55f"
server_key_version: "encryptionkey@0"
I20260810 07:23:35.781817 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b"
format_stamp: "Formatted at 2026-08-10 07:23:35 on dist-test-slave-x5fp"
server_key: "3047e4f0508a869e1895c66808d0db68"
server_key_iv: "2a49199fa93e778f7f6c57cbb1d8e55f"
server_key_version: "encryptionkey@0"
I20260810 07:23:35.783185 9363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:23:35.783944 9388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:35.784142 9363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:23:35.784258 9363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b"
format_stamp: "Formatted at 2026-08-10 07:23:35 on dist-test-slave-x5fp"
server_key: "3047e4f0508a869e1895c66808d0db68"
server_key_iv: "2a49199fa93e778f7f6c57cbb1d8e55f"
server_key_version: "encryptionkey@0"
I20260810 07:23:35.784336 9363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:35.793764 9363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:35.796052 9363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:35.796221 9363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:35.800477 9363 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:45041
I20260810 07:23:35.800518 9465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:45041 every 8 connection(s)
I20260810 07:23:35.800943 9363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:35.801913 9466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:35.804452 9466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b: Bootstrap starting.
I20260810 07:23:35.805135 9466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:35.805410 9466 log.cc:824] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:35.806159 9466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b: No bootstrap required, opened a new log
I20260810 07:23:35.806998 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 9363
I20260810 07:23:35.807127 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:35.807353 14992 external_mini_cluster.cc:1428] Setting key 1a6dceda7aa0acb432bfec4222faf142
I20260810 07:23:35.808547 9466 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45041 } }
I20260810 07:23:35.808774 9466 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:35.808844 9466 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a1ceafe0e14dfabbc7db5b26152e4b, State: Initialized, Role: FOLLOWER
I20260810 07:23:35.808990 9466 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45041 } }
I20260810 07:23:35.809098 9466 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:35.809139 9466 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:35.809217 9466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:35.810290 9466 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45041 } }
I20260810 07:23:35.810422 9466 leader_election.cc:302] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8a1ceafe0e14dfabbc7db5b26152e4b; no voters:
I20260810 07:23:35.810644 9466 leader_election.cc:288] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:35.810689 9474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:35.810882 9474 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [term 1 LEADER]: Becoming Leader. State: Replica: e8a1ceafe0e14dfabbc7db5b26152e4b, State: Running, Role: LEADER
I20260810 07:23:35.810971 9466 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:35.811028 9474 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45041 } }
I20260810 07:23:35.811447 9474 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8a1ceafe0e14dfabbc7db5b26152e4b. Latest consensus state: current_term: 1 leader_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45041 } } }
I20260810 07:23:35.811687 9474 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:35.811609 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a1ceafe0e14dfabbc7db5b26152e4b" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 45041 } } }
I20260810 07:23:35.812101 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:35.812211 9494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:35.812950 9494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:23:35 dist-test-slave-x5fp krb5kdc[9346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:35.819373 9494 catalog_manager.cc:1347] Generated new cluster ID: d9446ad2e2314baa9c48800318745df1
I20260810 07:23:35.819511 9494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:35.819859 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:35.807932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.14.164.62:45041)
0810 07:23:35.808103 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:35.808111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:35.808137 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:23:35.808523 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:35.808530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:35.808577 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:35.808688 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:35.809621 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:35.810754 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:35.812028 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:35.812692 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:35.812865 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:35.815976 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:35.815980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:35.815981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:35.815992 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:35.818001 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:35.818430 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:35.818433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:35.818434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:35.818572 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:35.818857 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:35.818858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:35.818858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:35.819138 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:35.819291 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:35.819478 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 07:23:35.819644 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260810 07:23:35.824317 9494 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:35.824965 9494 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:35.833397 9494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b: Generated new TSK 0
I20260810 07:23:35.833587 9494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:23:35.871572 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45041
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:35.962754 9514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:35.963071 9514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:35.963115 9514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:35.964308 9514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:35.964375 9514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:35.964422 9514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:35.966030 9514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:35.966355 9514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:35.966655 9514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:35.972913 9526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:35.973031 9527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:35.972919 9531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:35.973220 9514 server_base.cc:1034] running on GCE node
I20260810 07:23:35.974087 9514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:35.974377 9514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:35.975539 9514 hybrid_clock.cc:648] HybridClock initialized: now 1786346615975520 us; error 41 us; skew 500 ppm
Aug 10 07:23:35 dist-test-slave-x5fp krb5kdc[9346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346615, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:35.978318 9514 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:23:35.978747 9514 webserver.cc:460] Webserver started at http://127.14.164.1:34683/ using document root <none> and password file <none>
I20260810 07:23:35.978993 9514 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:35.979041 9514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:35.979183 9514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:35.980355 9514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "8447385b2d4a43eaa80d77157018b7a6"
format_stamp: "Formatted at 2026-08-10 07:23:35 on dist-test-slave-x5fp"
server_key: "7f14e08bcdc8902ed984b86efe50a480"
server_key_iv: "039246724e69747dca2a8c99b0aaead8"
server_key_version: "encryptionkey@0"
I20260810 07:23:35.980764 9514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "8447385b2d4a43eaa80d77157018b7a6"
format_stamp: "Formatted at 2026-08-10 07:23:35 on dist-test-slave-x5fp"
server_key: "7f14e08bcdc8902ed984b86efe50a480"
server_key_iv: "039246724e69747dca2a8c99b0aaead8"
server_key_version: "encryptionkey@0"
I20260810 07:23:35.982172 9514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:35.982966 9541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:35.983152 9514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:35.983268 9514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "8447385b2d4a43eaa80d77157018b7a6"
format_stamp: "Formatted at 2026-08-10 07:23:35 on dist-test-slave-x5fp"
server_key: "7f14e08bcdc8902ed984b86efe50a480"
server_key_iv: "039246724e69747dca2a8c99b0aaead8"
server_key_version: "encryptionkey@0"
I20260810 07:23:35.983347 9514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:35.992254 9514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:35.994629 9514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:35.994799 9514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:35.995055 9514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:35.995386 9514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:35.995443 9514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:35.995486 9514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:35.995512 9514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.001866 9514 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:37875
I20260810 07:23:36.001914 9708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:37875 every 8 connection(s)
I20260810 07:23:36.002245 9514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:23:36 dist-test-slave-x5fp krb5kdc[9346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346615, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:36.008845 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 9514
I20260810 07:23:36.008996 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:36.009264 14992 external_mini_cluster.cc:1428] Setting key 553ecaa1e7e2ba04f3ae9244d47a8eaa
I20260810 07:23:36.013294 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.003004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:46405 (local address 127.14.164.62:45041)
0810 07:23:36.003074 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.003076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.003268 (+ 192us) server_negotiation.cc:407] Connection header received
0810 07:23:36.003771 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.003772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.003782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.003838 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:36.004879 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.005411 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.006442 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.006665 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.006733 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.009278 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:36.009280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:36.009280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:36.009292 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:36.011216 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.011749 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.011750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.011751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.011830 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.012367 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.012368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.012369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.012502 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:36.012561 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.013116 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.013207 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:23:36.013588 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.003000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45041 (local address 127.14.164.1:46405)
0810 07:23:36.003160 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.003198 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.003682 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.003923 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.003926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.004512 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:36.004763 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.004771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.005493 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.005494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.006334 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.006336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.006780 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.006781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.006897 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.007510 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:36.007518 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:36.009127 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:36.011312 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:36.011321 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:36.011324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:36.011667 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:36.012074 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:36.012075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:36.012076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:36.012276 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:36.012587 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:36.012596 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:36.012862 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.013031 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.013270 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260810 07:23:36.014322 9710 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45041
I20260810 07:23:36.014459 9710 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:36.014688 9710 heartbeater.cc:511] Master 127.14.164.62:45041 requested a full tablet report, sending...
I20260810 07:23:36.015327 9416 ts_manager.cc:194] Registered new tserver with Master: 8447385b2d4a43eaa80d77157018b7a6 (127.14.164.1:37875)
I20260810 07:23:36.016206 9416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:46405
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:23:36.041013 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45041
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:36.126793 9717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:36.127058 9717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:36.127099 9717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:36.128269 9717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:36.128343 9717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:36.128386 9717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:36.129894 9717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:36.130175 9717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:36.130416 9717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:36.135643 9732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:36.135653 9733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:36.135648 9737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:36.135860 9717 server_base.cc:1034] running on GCE node
I20260810 07:23:36.136130 9717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:36.136417 9717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:36.137575 9717 hybrid_clock.cc:648] HybridClock initialized: now 1786346616137564 us; error 27 us; skew 500 ppm
Aug 10 07:23:36 dist-test-slave-x5fp krb5kdc[9346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346616, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:36.140172 9717 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:23:36.140618 9717 webserver.cc:460] Webserver started at http://127.14.164.2:38489/ using document root <none> and password file <none>
I20260810 07:23:36.140870 9717 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:36.140920 9717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:36.141076 9717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:36.142166 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "3cfd70ecd80041739570619d7c57eaaf"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "3626bfa27f9480f7dd35167d218afdb0"
server_key_iv: "e92dd2e6b680e9d8cb81d6371bda7c6c"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.142553 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "3cfd70ecd80041739570619d7c57eaaf"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "3626bfa27f9480f7dd35167d218afdb0"
server_key_iv: "e92dd2e6b680e9d8cb81d6371bda7c6c"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.144052 9717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:36.145128 9753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.145533 9717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:36.145648 9717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "3cfd70ecd80041739570619d7c57eaaf"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "3626bfa27f9480f7dd35167d218afdb0"
server_key_iv: "e92dd2e6b680e9d8cb81d6371bda7c6c"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.145738 9717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:36.174939 9717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:36.177793 9717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:36.177974 9717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:36.178226 9717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:36.178574 9717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:36.178632 9717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.178684 9717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:36.178711 9717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.184775 9717 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:46813
I20260810 07:23:36.184842 9923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:46813 every 8 connection(s)
I20260810 07:23:36.185216 9717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:36.187386 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 9717
I20260810 07:23:36.187554 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:36.187783 14992 external_mini_cluster.cc:1428] Setting key 1c0c958855beaaddf71f3c570ba0d79a
Aug 10 07:23:36 dist-test-slave-x5fp krb5kdc[9346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346616, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:36.196345 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.186044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:59941 (local address 127.14.164.62:45041)
0810 07:23:36.186149 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.186151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.186386 (+ 235us) server_negotiation.cc:407] Connection header received
0810 07:23:36.187005 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.187007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.187021 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.187091 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:36.188348 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.188943 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.190024 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.190224 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.190305 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.193092 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:36.193093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:36.193093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:36.193100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:36.194563 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.195178 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.195178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.195179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.195232 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.195591 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.195592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.195592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.195706 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:36.195757 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.196191 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.196279 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:23:36.196661 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.186001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45041 (local address 127.14.164.2:59941)
0810 07:23:36.186266 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.186302 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.186885 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.187165 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.187169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.187980 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:36.188242 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.188248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.189043 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.189044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.189911 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.189913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.190322 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.190323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.190461 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.191096 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:36.191107 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:36.192983 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:36.194661 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:36.194670 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:36.194672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:36.195080 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:36.195313 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:36.195314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:36.195314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:36.195497 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:36.195784 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:36.195792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:36.195996 (+ 204us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.196154 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.196372 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":97,"threads_started":1}
I20260810 07:23:36.197539 9924 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45041
I20260810 07:23:36.197664 9924 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:36.197860 9924 heartbeater.cc:511] Master 127.14.164.62:45041 requested a full tablet report, sending...
I20260810 07:23:36.198371 9416 ts_manager.cc:194] Registered new tserver with Master: 3cfd70ecd80041739570619d7c57eaaf (127.14.164.2:46813)
I20260810 07:23:36.198997 9416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:59941
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:23:36.220090 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45041
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:36.305948 9929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:36.306157 9929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:36.306180 9929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:36.307332 9929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:36.307375 9929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:36.307404 9929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:36.308910 9929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:36.309157 9929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:36.309404 9929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:36.315200 9942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:36.315227 9939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:36.315196 9945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:36.315409 9929 server_base.cc:1034] running on GCE node
I20260810 07:23:36.315807 9929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:36.316032 9929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:36.317193 9929 hybrid_clock.cc:648] HybridClock initialized: now 1786346616317152 us; error 46 us; skew 500 ppm
Aug 10 07:23:36 dist-test-slave-x5fp krb5kdc[9346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346616, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:36.320011 9929 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:23:36.320478 9929 webserver.cc:460] Webserver started at http://127.14.164.3:41365/ using document root <none> and password file <none>
I20260810 07:23:36.320713 9929 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:36.320837 9929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:36.321003 9929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:36.322162 9929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "1b3c92c815b548b6ae11b1282be56a22"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "7a441b5cdf68d491d6444b55182a87dc"
server_key_iv: "8b5f8a75184f9003e42088d301e568f3"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.322563 9929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "1b3c92c815b548b6ae11b1282be56a22"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "7a441b5cdf68d491d6444b55182a87dc"
server_key_iv: "8b5f8a75184f9003e42088d301e568f3"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.324007 9929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:36.324965 9956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.325163 9929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:36.325275 9929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "1b3c92c815b548b6ae11b1282be56a22"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "7a441b5cdf68d491d6444b55182a87dc"
server_key_iv: "8b5f8a75184f9003e42088d301e568f3"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.325367 9929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:36.339783 9929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:36.342459 9929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:36.342641 9929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:36.342916 9929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:36.343261 9929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:36.343318 9929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.343362 9929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:36.343395 9929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.350131 9929 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:41431
I20260810 07:23:36.350188 10133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:41431 every 8 connection(s)
I20260810 07:23:36.350593 9929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:36.356384 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 9929
I20260810 07:23:36.356509 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:36.356763 14992 external_mini_cluster.cc:1428] Setting key 506e3176f542febbfc6e617f3200adf6
Aug 10 07:23:36 dist-test-slave-x5fp krb5kdc[9346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346616, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:36.361316 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.351356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:59795 (local address 127.14.164.62:45041)
0810 07:23:36.351460 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.351464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.351681 (+ 217us) server_negotiation.cc:407] Connection header received
0810 07:23:36.352255 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.352257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.352268 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.352330 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:36.353477 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.353987 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.355171 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.355388 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.355453 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.358188 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:36.358189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:36.358190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:36.358197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:36.359731 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.360185 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.360186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.360186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.360247 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.360601 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.360602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.360602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.360790 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:36.360859 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.361175 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.361247 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:23:36.361578 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.351326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:45041 (local address 127.14.164.3:59795)
0810 07:23:36.351543 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.351585 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.352122 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.352417 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.352420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.353145 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:36.353352 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.353358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.354138 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.354139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.355061 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.355064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.355503 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.355504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.355636 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.356365 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:36.356375 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:36.358075 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:36.359826 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:36.359834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:36.359836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:36.360095 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:36.360346 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:36.360347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:36.360347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:36.360483 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:36.360852 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:36.360858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:36.361040 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.361136 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.361286 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":126,"threads_started":1}
I20260810 07:23:36.362309 10134 heartbeater.cc:347] Connected to a master server at 127.14.164.62:45041
I20260810 07:23:36.362452 10134 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:36.362694 10134 heartbeater.cc:511] Master 127.14.164.62:45041 requested a full tablet report, sending...
I20260810 07:23:36.363368 9416 ts_manager.cc:194] Registered new tserver with Master: 1b3c92c815b548b6ae11b1282be56a22 (127.14.164.3:41431)
I20260810 07:23:36.363919 9416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:59795
I20260810 07:23:36.368955 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:36.377724 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.370511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.14.164.62:45041)
0810 07:23:36.370595 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.370597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.370702 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:23:36.370788 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.370789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.370800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.370870 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:36.371499 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.372052 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.372872 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.373087 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.373167 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.374464 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:36.374466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:36.374466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:36.374476 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:36.376050 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.376566 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.376567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.376568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.376642 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:36.377031 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:36.377032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:36.377033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:36.377225 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:36.377349 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.377493 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.377591 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:23:36.379805 9416 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:36.381747 9416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:36.388983 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.384826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.14.164.3:41431)
0810 07:23:36.385395 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.385400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.385409 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:36.385453 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.385454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.385470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.385567 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:36.385740 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.386906 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.387967 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.388587 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.388659 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.388765 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.388863 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.388905 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":90,"threads_started":1}
I20260810 07:23:36.388983 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.384455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:41431 (local address 127.0.0.1:47436)
0810 07:23:36.384860 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.384873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.384942 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.385581 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.385582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.385584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:36.385623 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.385625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.387013 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.387014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.387885 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.387890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.388678 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.388679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.388777 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.388782 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.388846 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.388891 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":75,"threads_started":1}
I20260810 07:23:36.389070 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.384626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.14.164.2:46813)
0810 07:23:36.384820 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.384824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.384850 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:23:36.384900 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.384903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.384920 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.385205 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:36.385288 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.386443 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.387898 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.388477 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.388536 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.388578 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.388939 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.388982 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260810 07:23:36.389436 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.384237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:46813 (local address 127.0.0.1:59048)
0810 07:23:36.384430 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.384479 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.384554 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.385038 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.385041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.385044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:36.385125 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.385128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.386541 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.386542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.387800 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.387803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.388559 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.388560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.388649 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.388666 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.388771 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.388839 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":63,"thread_start_us":71,"threads_started":1}
I20260810 07:23:36.390044 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.384422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.14.164.1:37875)
0810 07:23:36.385071 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.385076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.385086 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:36.385127 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.385129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.385143 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.385225 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:36.385350 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.386174 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.388611 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.389553 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.389630 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.389693 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.389913 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.389970 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":98,"threads_started":1}
I20260810 07:23:36.390002 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.384259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:37875 (local address 127.0.0.1:33364)
0810 07:23:36.384613 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.384623 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.384665 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.385225 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.385226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.385228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:36.385268 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.385270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.386288 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.386290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.387802 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.387804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.389727 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.389728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.389792 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.389814 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.389892 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.389934 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":68,"thread_start_us":89,"threads_started":1}
I20260810 07:23:36.389953 10030 tablet_service.cc:1467] Processing CreateTablet for tablet dcd3ee636ecb45059d350c1f1320b2c3 (DEFAULT_TABLE table=test-table [id=98a9ca510df1426cbf2509dc7737c4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:36.390369 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd3ee636ecb45059d350c1f1320b2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:36.390352 9817 tablet_service.cc:1467] Processing CreateTablet for tablet dcd3ee636ecb45059d350c1f1320b2c3 (DEFAULT_TABLE table=test-table [id=98a9ca510df1426cbf2509dc7737c4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:36.390687 9817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd3ee636ecb45059d350c1f1320b2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:36.393505 10159 tablet_bootstrap.cc:492] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Bootstrap starting.
I20260810 07:23:36.393471 10158 tablet_bootstrap.cc:492] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Bootstrap starting.
I20260810 07:23:36.394222 10159 tablet_bootstrap.cc:654] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:36.394447 10158 tablet_bootstrap.cc:654] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:36.394354 9612 tablet_service.cc:1467] Processing CreateTablet for tablet dcd3ee636ecb45059d350c1f1320b2c3 (DEFAULT_TABLE table=test-table [id=98a9ca510df1426cbf2509dc7737c4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:36.394712 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd3ee636ecb45059d350c1f1320b2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:36.394855 10159 log.cc:824] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:36.395047 10158 log.cc:824] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:36.395695 10159 tablet_bootstrap.cc:492] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: No bootstrap required, opened a new log
I20260810 07:23:36.395808 10159 ts_tablet_manager.cc:1397] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:36.395936 10158 tablet_bootstrap.cc:492] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: No bootstrap required, opened a new log
I20260810 07:23:36.396040 10158 ts_tablet_manager.cc:1397] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:23:36.397419 10159 raft_consensus.cc:348] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.397598 10159 raft_consensus.cc:374] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:36.397662 10159 raft_consensus.cc:729] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cfd70ecd80041739570619d7c57eaaf, State: Initialized, Role: FOLLOWER
I20260810 07:23:36.397872 10159 consensus_queue.cc:260] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.398224 10159 ts_tablet_manager.cc:1428] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:36.398236 10163 tablet_bootstrap.cc:492] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: Bootstrap starting.
I20260810 07:23:36.398180 10158 raft_consensus.cc:348] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.398455 9924 heartbeater.cc:503] Master 127.14.164.62:45041 was elected leader, sending a full tablet report...
I20260810 07:23:36.398444 10158 raft_consensus.cc:374] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:36.398500 10158 raft_consensus.cc:729] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3c92c815b548b6ae11b1282be56a22, State: Initialized, Role: FOLLOWER
I20260810 07:23:36.398610 10158 consensus_queue.cc:260] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.398890 10158 ts_tablet_manager.cc:1428] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:36.399027 10134 heartbeater.cc:503] Master 127.14.164.62:45041 was elected leader, sending a full tablet report...
I20260810 07:23:36.398926 10163 tablet_bootstrap.cc:654] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:36.399935 10163 log.cc:824] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:36.400818 10163 tablet_bootstrap.cc:492] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: No bootstrap required, opened a new log
I20260810 07:23:36.400898 10163 ts_tablet_manager.cc:1397] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:23:36.402240 10163 raft_consensus.cc:348] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.402411 10163 raft_consensus.cc:374] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:36.402459 10163 raft_consensus.cc:729] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8447385b2d4a43eaa80d77157018b7a6, State: Initialized, Role: FOLLOWER
I20260810 07:23:36.402570 10163 consensus_queue.cc:260] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.402842 10163 ts_tablet_manager.cc:1428] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:36.402884 9710 heartbeater.cc:503] Master 127.14.164.62:45041 was elected leader, sending a full tablet report...
W20260810 07:23:36.435940 9925 tablet.cc:2369] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:36.470036 10169 raft_consensus.cc:482] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:36.470187 10169 raft_consensus.cc:504] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.470701 10169 leader_election.cc:288] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cfd70ecd80041739570619d7c57eaaf (127.14.164.2:46813), 1b3c92c815b548b6ae11b1282be56a22 (127.14.164.3:41431)
I20260810 07:23:36.474032 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.471103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:46921 (local address 127.14.164.2:46813)
0810 07:23:36.471150 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.471151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.471369 (+ 218us) server_negotiation.cc:407] Connection header received
0810 07:23:36.471460 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.471461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.471476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.471545 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:36.471720 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.472189 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.473242 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.473641 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.473729 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.473738 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.473904 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.473948 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:23:36.474040 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.471040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:45513 (local address 127.14.164.3:41431)
0810 07:23:36.471103 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.471105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.471120 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:23:36.471163 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.471164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.471176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.471278 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:36.471504 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.472069 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.473156 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.473604 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.473702 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.473714 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.473904 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.473953 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:23:36.474076 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.470953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:41431 (local address 127.14.164.1:45513)
0810 07:23:36.471017 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.471044 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.471095 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.471306 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.471307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.471310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:36.471413 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.471415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.472172 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.472173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.473046 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.473048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.473700 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.473701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.473810 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.473814 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.473889 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.473947 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":4}
I20260810 07:23:36.474086 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.470957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:46813 (local address 127.14.164.1:46921)
0810 07:23:36.471304 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.471315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.471396 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.471550 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.471551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.471553 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:36.471634 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.471635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.472279 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.472280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.473142 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.473144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.473743 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.473744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.473826 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.473829 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.473894 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.473941 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":17,"thread_start_us":268,"threads_started":1}
I20260810 07:23:36.474587 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd3ee636ecb45059d350c1f1320b2c3" candidate_uuid: "8447385b2d4a43eaa80d77157018b7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b3c92c815b548b6ae11b1282be56a22" is_pre_election: true
I20260810 07:23:36.474615 9852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd3ee636ecb45059d350c1f1320b2c3" candidate_uuid: "8447385b2d4a43eaa80d77157018b7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cfd70ecd80041739570619d7c57eaaf" is_pre_election: true
I20260810 07:23:36.474776 9852 raft_consensus.cc:2454] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8447385b2d4a43eaa80d77157018b7a6 in term 0.
I20260810 07:23:36.474776 10061 raft_consensus.cc:2454] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8447385b2d4a43eaa80d77157018b7a6 in term 0.
I20260810 07:23:36.475116 9543 leader_election.cc:302] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cfd70ecd80041739570619d7c57eaaf, 8447385b2d4a43eaa80d77157018b7a6; no voters:
I20260810 07:23:36.475334 10169 raft_consensus.cc:2781] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:36.475446 10169 raft_consensus.cc:482] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:36.475486 10169 raft_consensus.cc:3037] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:36.476641 10169 raft_consensus.cc:504] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.476981 10169 leader_election.cc:288] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [CANDIDATE]: Term 1 election: Requested vote from peers 3cfd70ecd80041739570619d7c57eaaf (127.14.164.2:46813), 1b3c92c815b548b6ae11b1282be56a22 (127.14.164.3:41431)
I20260810 07:23:36.477070 9852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd3ee636ecb45059d350c1f1320b2c3" candidate_uuid: "8447385b2d4a43eaa80d77157018b7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cfd70ecd80041739570619d7c57eaaf"
I20260810 07:23:36.477185 9852 raft_consensus.cc:3037] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:36.477322 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd3ee636ecb45059d350c1f1320b2c3" candidate_uuid: "8447385b2d4a43eaa80d77157018b7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b3c92c815b548b6ae11b1282be56a22"
I20260810 07:23:36.477423 10061 raft_consensus.cc:3037] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:36.478037 9852 raft_consensus.cc:2454] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8447385b2d4a43eaa80d77157018b7a6 in term 1.
I20260810 07:23:36.478268 10061 raft_consensus.cc:2454] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8447385b2d4a43eaa80d77157018b7a6 in term 1.
I20260810 07:23:36.478281 9543 leader_election.cc:302] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cfd70ecd80041739570619d7c57eaaf, 8447385b2d4a43eaa80d77157018b7a6; no voters:
I20260810 07:23:36.478439 10169 raft_consensus.cc:2781] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:36.478655 10169 raft_consensus.cc:686] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [term 1 LEADER]: Becoming Leader. State: Replica: 8447385b2d4a43eaa80d77157018b7a6, State: Running, Role: LEADER
I20260810 07:23:36.478787 10169 consensus_queue.cc:237] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } }
I20260810 07:23:36.479830 9415 catalog_manager.cc:5495] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8447385b2d4a43eaa80d77157018b7a6 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "8447385b2d4a43eaa80d77157018b7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8447385b2d4a43eaa80d77157018b7a6" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 37875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 } health_report { overall_health: UNKNOWN } } }
W20260810 07:23:36.503029 9711 tablet.cc:2369] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:36.505864 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.503192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.14.164.1:37875)
0810 07:23:36.503275 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.503277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.503291 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:23:36.503334 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.503335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.503362 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.503475 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:36.503685 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.504170 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.505056 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.505276 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.505339 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.505521 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:36.505626 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:36.505691 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.505751 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.505774 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:23:36.507803 9852 raft_consensus.cc:1261] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 1 FOLLOWER]: Refusing update from remote peer 8447385b2d4a43eaa80d77157018b7a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:36.507822 10061 raft_consensus.cc:1261] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 1 FOLLOWER]: Refusing update from remote peer 8447385b2d4a43eaa80d77157018b7a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:36.508189 10173 consensus_queue.cc:1035] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cfd70ecd80041739570619d7c57eaaf" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:36.508373 10169 consensus_queue.cc:1035] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b3c92c815b548b6ae11b1282be56a22" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:36.514226 9415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:36.514345 9415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:36.515182 9415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8a1ceafe0e14dfabbc7db5b26152e4b: Sending DeleteTablet for 3 replicas of tablet dcd3ee636ecb45059d350c1f1320b2c3
I20260810 07:23:36.515506 9612 tablet_service.cc:1514] Processing DeleteTablet for tablet dcd3ee636ecb45059d350c1f1320b2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:36 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:33364
I20260810 07:23:36.515578 10030 tablet_service.cc:1514] Processing DeleteTablet for tablet dcd3ee636ecb45059d350c1f1320b2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:36 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:47436
I20260810 07:23:36.515761 9817 tablet_service.cc:1514] Processing DeleteTablet for tablet dcd3ee636ecb45059d350c1f1320b2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:36 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59048
I20260810 07:23:36.515864 10199 tablet_replica.cc:330] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: stopping tablet replica
I20260810 07:23:36.515991 10198 tablet_replica.cc:330] T dcd3ee636ecb45059d350c1f1320b2c3 P 8447385b2d4a43eaa80d77157018b7a6: stopping tablet replica
I20260810 07:23:36.516002 10199 raft_consensus.cc:2229] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:36.516188 10201 tablet_replica.cc:330] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: stopping tablet replica
I20260810 07:23:36.516213 10199 raft_consensus.cc:2258] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:36.516393 10201 raft_consensus.cc:2229] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:36.516459 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 9514
I20260810 07:23:36.517320 10199 ts_tablet_manager.cc:1905] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:36.520083 10199 ts_tablet_manager.cc:1918] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:36.520203 10199 log.cc:1196] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/dcd3ee636ecb45059d350c1f1320b2c3
I20260810 07:23:36.520654 10199 ts_tablet_manager.cc:1939] T dcd3ee636ecb45059d350c1f1320b2c3 P 1b3c92c815b548b6ae11b1282be56a22: Deleting consensus metadata
I20260810 07:23:36.522634 9392 catalog_manager.cc:4841] TS 1b3c92c815b548b6ae11b1282be56a22 (127.14.164.3:41431): tablet dcd3ee636ecb45059d350c1f1320b2c3 (table test-table [id=98a9ca510df1426cbf2509dc7737c4d3]) successfully deleted
I20260810 07:23:36.524691 10201 raft_consensus.cc:2258] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:36.530772 10201 ts_tablet_manager.cc:1905] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:36.533352 10201 ts_tablet_manager.cc:1918] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:36.533475 10201 log.cc:1196] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/dcd3ee636ecb45059d350c1f1320b2c3
I20260810 07:23:36.533849 10201 ts_tablet_manager.cc:1939] T dcd3ee636ecb45059d350c1f1320b2c3 P 3cfd70ecd80041739570619d7c57eaaf: Deleting consensus metadata
I20260810 07:23:36.534474 9390 catalog_manager.cc:4841] TS 3cfd70ecd80041739570619d7c57eaaf (127.14.164.2:46813): tablet dcd3ee636ecb45059d350c1f1320b2c3 (table test-table [id=98a9ca510df1426cbf2509dc7737c4d3]) successfully deleted
W20260810 07:23:36.536267 9392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:37875 (error 108)
I20260810 07:23:36.536921 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.536749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:37875 (local address 127.0.0.1:33384)
0810 07:23:36.536816 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.536865 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:23:36.537053 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 9717
W20260810 07:23:36.537088 9392 catalog_manager.cc:4584] TS 8447385b2d4a43eaa80d77157018b7a6 (127.14.164.1:37875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dcd3ee636ecb45059d350c1f1320b2c3: Network error: Client connection negotiation failed: client connection to 127.14.164.1:37875: connect: Connection refused (error 111)
I20260810 07:23:36.555505 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 9929
I20260810 07:23:36.568878 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 9363
2026-08-10T07:23:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2290 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T07:23:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:36Z Disabled control of system clock
I20260810 07:23:36.610734 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:43755
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:43755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:23:36.696707 10209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:36.697010 10209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:36.697032 10209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:36.698225 10209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:36.698273 10209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:36.698288 10209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:36.698302 10209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:36.698313 10209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:36.699707 10209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:43755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:43755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:36.699975 10209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:36.700296 10209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:36.705472 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:36.705503 10209 server_base.cc:1034] running on GCE node
W20260810 07:23:36.705502 10216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:36.705472 10218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:36.706043 10209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:36.706346 10209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:36.707508 10209 hybrid_clock.cc:648] HybridClock initialized: now 1786346616707486 us; error 60 us; skew 500 ppm
I20260810 07:23:36.708976 10209 webserver.cc:460] Webserver started at http://127.14.164.62:40269/ using document root <none> and password file <none>
I20260810 07:23:36.709270 10209 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:36.709319 10209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:36.709465 10209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:36.710577 10209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "867ac9e9d2934dbb8fabd244c3341feb"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "d3afcc03cb8f75d00c5560e89fdbc5d8"
server_key_iv: "03614228b4cfa7e7f5154ba69a1ccb74"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.710958 10209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "867ac9e9d2934dbb8fabd244c3341feb"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "d3afcc03cb8f75d00c5560e89fdbc5d8"
server_key_iv: "03614228b4cfa7e7f5154ba69a1ccb74"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.712347 10209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:36.713289 10228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.713505 10209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:36.713618 10209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "867ac9e9d2934dbb8fabd244c3341feb"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "d3afcc03cb8f75d00c5560e89fdbc5d8"
server_key_iv: "03614228b4cfa7e7f5154ba69a1ccb74"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.713696 10209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:36.730598 10209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:36.730986 10209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:36.731159 10209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:36.735430 10209 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:43755
I20260810 07:23:36.735471 10305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:43755 every 8 connection(s)
I20260810 07:23:36.735867 10209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:36.736251 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 10209
I20260810 07:23:36.736395 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:36.736618 14992 external_mini_cluster.cc:1428] Setting key f985e629e1a55ffa267f4ac2b5f1eff2
I20260810 07:23:36.737087 10306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:36.740448 10306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb: Bootstrap starting.
I20260810 07:23:36.741168 10306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:36.741513 10306 log.cc:824] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:36.742470 10306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb: No bootstrap required, opened a new log
I20260810 07:23:36.743116 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.737245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.14.164.62:43755)
0810 07:23:36.737468 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.737477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.737509 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:23:36.737956 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.737962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.738025 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.738140 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:36.738449 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.739503 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.740640 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.741189 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.741358 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.742336 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:36.742340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:36.742340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:36.742538 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:36.742581 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:36.742603 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:36.742674 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.742754 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.742946 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260810 07:23:36.744696 10306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "867ac9e9d2934dbb8fabd244c3341feb" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43755 } }
I20260810 07:23:36.744908 10306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:36.744933 10306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867ac9e9d2934dbb8fabd244c3341feb, State: Initialized, Role: FOLLOWER
I20260810 07:23:36.745081 10306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "867ac9e9d2934dbb8fabd244c3341feb" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43755 } }
I20260810 07:23:36.745169 10306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:36.745203 10306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:36.745270 10306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:36.745913 10306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "867ac9e9d2934dbb8fabd244c3341feb" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43755 } }
I20260810 07:23:36.746007 10306 leader_election.cc:302] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 867ac9e9d2934dbb8fabd244c3341feb; no voters:
I20260810 07:23:36.746238 10306 leader_election.cc:288] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:36.746528 10312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:36.746697 10312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [term 1 LEADER]: Becoming Leader. State: Replica: 867ac9e9d2934dbb8fabd244c3341feb, State: Running, Role: LEADER
I20260810 07:23:36.746788 10306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:36.746878 10312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "867ac9e9d2934dbb8fabd244c3341feb" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43755 } }
I20260810 07:23:36.747301 10314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "867ac9e9d2934dbb8fabd244c3341feb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "867ac9e9d2934dbb8fabd244c3341feb" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43755 } } }
I20260810 07:23:36.747411 10314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:36.747313 10317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867ac9e9d2934dbb8fabd244c3341feb. Latest consensus state: current_term: 1 leader_uuid: "867ac9e9d2934dbb8fabd244c3341feb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "867ac9e9d2934dbb8fabd244c3341feb" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 43755 } } }
I20260810 07:23:36.747545 10317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:36.747783 10325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:36.748340 10325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:36.749895 10325 catalog_manager.cc:1347] Generated new cluster ID: 0913b082c92444e99de79c889fb3c04b
I20260810 07:23:36.749971 10325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:36.758939 10325 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:36.759687 10325 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:36.771241 10325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb: Generated new TSK 0
I20260810 07:23:36.771550 10325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:36.773262 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43755
--builtin_ntp_servers=127.14.164.20:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:23:36.879331 10341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:36.879582 10341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:36.879624 10341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:36.880936 10341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:36.880997 10341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:36.881039 10341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:36.882436 10341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:36.882705 10341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:36.883026 10341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:36.889339 10353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:36.889309 10348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:36.889516 10341 server_base.cc:1034] running on GCE node
W20260810 07:23:36.889338 10347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:36.889863 10341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:36.890177 10341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:36.891369 10341 hybrid_clock.cc:648] HybridClock initialized: now 1786346616891353 us; error 32 us; skew 500 ppm
I20260810 07:23:36.893020 10341 webserver.cc:460] Webserver started at http://127.14.164.1:46057/ using document root <none> and password file <none>
I20260810 07:23:36.893361 10341 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:36.893458 10341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:36.893682 10341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:36.894945 10341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "8efcb15cbcb441c6842123c6d71c625b"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "f156f047fbe4a1e8b3f342dc80579297"
server_key_iv: "e3c62f638b67f38611421dde4db4131c"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.895427 10341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "8efcb15cbcb441c6842123c6d71c625b"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "f156f047fbe4a1e8b3f342dc80579297"
server_key_iv: "e3c62f638b67f38611421dde4db4131c"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.897101 10341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 07:23:36.898058 10361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.898293 10341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:36.898416 10341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "8efcb15cbcb441c6842123c6d71c625b"
format_stamp: "Formatted at 2026-08-10 07:23:36 on dist-test-slave-x5fp"
server_key: "f156f047fbe4a1e8b3f342dc80579297"
server_key_iv: "e3c62f638b67f38611421dde4db4131c"
server_key_version: "encryptionkey@0"
I20260810 07:23:36.898501 10341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:36.910967 10341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:36.911309 10341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:36.911465 10341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:36.911721 10341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:36.912075 10341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:36.912133 10341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.912178 10341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:36.912207 10341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:36.918392 10341 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:44235
I20260810 07:23:36.918442 10537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:44235 every 8 connection(s)
I20260810 07:23:36.918819 10341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:36.920751 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 10341
I20260810 07:23:36.920871 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:36.921079 14992 external_mini_cluster.cc:1428] Setting key db7cda6dd1ce8bc299d968f6aa7db8bd
I20260810 07:23:36.922122 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43755
--builtin_ntp_servers=127.14.164.20:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:23:36.925148 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.919618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:34525 (local address 127.14.164.62:43755)
0810 07:23:36.919684 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:36.919686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:36.919950 (+ 264us) server_negotiation.cc:407] Connection header received
0810 07:23:36.920442 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:36.920443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:36.920457 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:36.920516 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:36.921448 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.922042 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.923137 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.923369 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.923447 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.924683 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:36.924685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:36.924685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:36.924712 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:36.924744 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:36.924749 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:36.924807 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:36.924994 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 07:23:36.925044 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:23:36.925638 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:36.919592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:43755 (local address 127.14.164.1:34525)
0810 07:23:36.919803 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:36.919846 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:36.920358 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:36.920575 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:36.920579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:36.920586 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:36.921347 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.921356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.922147 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.922148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.923041 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:36.923044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:36.923480 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:36.923481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:36.923661 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:36.924403 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:36.924403 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:36.924477 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:36.924509 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:36.924886 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:36.924894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:36.924897 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:36.925258 (+ 361us) client_negotiation.cc:240] Negotiation successful
0810 07:23:36.925392 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260810 07:23:36.927203 10538 heartbeater.cc:347] Connected to a master server at 127.14.164.62:43755
I20260810 07:23:36.927317 10538 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:36.927524 10538 heartbeater.cc:511] Master 127.14.164.62:43755 requested a full tablet report, sending...
I20260810 07:23:36.928140 10253 ts_manager.cc:194] Registered new tserver with Master: 8efcb15cbcb441c6842123c6d71c625b (127.14.164.1:44235)
I20260810 07:23:36.928838 10253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.1:34525
W20260810 07:23:37.009990 10543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:37.010221 10543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:37.010244 10543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:37.011359 10543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:37.011394 10543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:37.011418 10543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:37.012761 10543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:37.013013 10543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:37.013293 10543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:37.018633 10557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:37.018653 10543 server_base.cc:1034] running on GCE node
W20260810 07:23:37.018771 10554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:37.018640 10552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:37.019120 10543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:37.019395 10543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:37.020566 10543 hybrid_clock.cc:648] HybridClock initialized: now 1786346617020544 us; error 46 us; skew 500 ppm
I20260810 07:23:37.021994 10543 webserver.cc:460] Webserver started at http://127.14.164.2:43573/ using document root <none> and password file <none>
I20260810 07:23:37.022280 10543 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:37.022356 10543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:37.022521 10543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:37.023695 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "c7744c2be3d5454383ab081981c14b57"
format_stamp: "Formatted at 2026-08-10 07:23:37 on dist-test-slave-x5fp"
server_key: "12651186d6444dd4c98e46782e3cd775"
server_key_iv: "66d02b7443f41c627315783cc0e28817"
server_key_version: "encryptionkey@0"
I20260810 07:23:37.024091 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "c7744c2be3d5454383ab081981c14b57"
format_stamp: "Formatted at 2026-08-10 07:23:37 on dist-test-slave-x5fp"
server_key: "12651186d6444dd4c98e46782e3cd775"
server_key_iv: "66d02b7443f41c627315783cc0e28817"
server_key_version: "encryptionkey@0"
I20260810 07:23:37.025610 10543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:37.026472 10564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:37.026696 10543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:37.026811 10543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "c7744c2be3d5454383ab081981c14b57"
format_stamp: "Formatted at 2026-08-10 07:23:37 on dist-test-slave-x5fp"
server_key: "12651186d6444dd4c98e46782e3cd775"
server_key_iv: "66d02b7443f41c627315783cc0e28817"
server_key_version: "encryptionkey@0"
I20260810 07:23:37.026908 10543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:37.061122 10543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:37.061488 10543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:37.061659 10543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:37.061929 10543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:37.062289 10543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:37.062345 10543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:37.062381 10543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:37.062414 10543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:37.068295 10543 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:33105
I20260810 07:23:37.068442 10731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:33105 every 8 connection(s)
I20260810 07:23:37.068804 10543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:37.069934 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 10543
I20260810 07:23:37.070037 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:37.070255 14992 external_mini_cluster.cc:1428] Setting key 384f3bacfc6e67fee3a46c520416fd5f
I20260810 07:23:37.071472 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43755
--builtin_ntp_servers=127.14.164.20:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:23:37.075580 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.069680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:36515 (local address 127.14.164.62:43755)
0810 07:23:37.069773 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.069775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.070017 (+ 242us) server_negotiation.cc:407] Connection header received
0810 07:23:37.070761 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.070762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.070774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.070836 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:37.071915 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.072423 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.073867 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.074191 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.074274 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.075196 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:37.075198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:37.075198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:37.075204 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:37.075224 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:37.075229 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:37.075289 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.075427 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.075457 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:23:37.075740 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.069627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:43755 (local address 127.14.164.2:36515)
0810 07:23:37.069849 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.069904 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.070647 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.070961 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.070966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.070977 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:37.071792 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.071799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.072551 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.072552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.073762 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.073765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.074285 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.074286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.074411 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.075036 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:37.075037 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:37.075069 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:37.075095 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:37.075324 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:37.075332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:37.075335 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.075412 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.075504 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260810 07:23:37.076332 10732 heartbeater.cc:347] Connected to a master server at 127.14.164.62:43755
I20260810 07:23:37.076457 10732 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:37.076676 10732 heartbeater.cc:511] Master 127.14.164.62:43755 requested a full tablet report, sending...
I20260810 07:23:37.077191 10253 ts_manager.cc:194] Registered new tserver with Master: c7744c2be3d5454383ab081981c14b57 (127.14.164.2:33105)
I20260810 07:23:37.077577 10253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.2:36515
W20260810 07:23:37.160671 10739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:37.160967 10739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:37.160991 10739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:37.162227 10739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:37.162273 10739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:37.162299 10739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:37.163772 10739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:43755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:37.164052 10739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:37.164405 10739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:37.170377 10746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:37.170316 10750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:37.170327 10745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:37.170661 10739 server_base.cc:1034] running on GCE node
I20260810 07:23:37.170909 10739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:37.171183 10739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:37.172369 10739 hybrid_clock.cc:648] HybridClock initialized: now 1786346617172357 us; error 31 us; skew 500 ppm
I20260810 07:23:37.173784 10739 webserver.cc:460] Webserver started at http://127.14.164.3:38633/ using document root <none> and password file <none>
I20260810 07:23:37.174070 10739 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:37.174147 10739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:37.174312 10739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:37.175432 10739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "e56fca4b4d5141dab0641b22dcbb342c"
format_stamp: "Formatted at 2026-08-10 07:23:37 on dist-test-slave-x5fp"
server_key: "d5155f94f541d20823b9424aa2f4539b"
server_key_iv: "4b9738869164ad0ffd31b1fd3166ae8f"
server_key_version: "encryptionkey@0"
I20260810 07:23:37.175817 10739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "e56fca4b4d5141dab0641b22dcbb342c"
format_stamp: "Formatted at 2026-08-10 07:23:37 on dist-test-slave-x5fp"
server_key: "d5155f94f541d20823b9424aa2f4539b"
server_key_iv: "4b9738869164ad0ffd31b1fd3166ae8f"
server_key_version: "encryptionkey@0"
I20260810 07:23:37.177362 10739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:37.178206 10757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:37.178519 10739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:37.178634 10739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "e56fca4b4d5141dab0641b22dcbb342c"
format_stamp: "Formatted at 2026-08-10 07:23:37 on dist-test-slave-x5fp"
server_key: "d5155f94f541d20823b9424aa2f4539b"
server_key_iv: "4b9738869164ad0ffd31b1fd3166ae8f"
server_key_version: "encryptionkey@0"
I20260810 07:23:37.178717 10739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:37.194433 10739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:37.194823 10739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:37.194999 10739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:37.195279 10739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:37.195653 10739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:37.195714 10739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:37.195757 10739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:37.195806 10739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:37.202461 10739 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:41433
I20260810 07:23:37.202548 10916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:41433 every 8 connection(s)
I20260810 07:23:37.202910 10739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:37.208763 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.203940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:42785 (local address 127.14.164.62:43755)
0810 07:23:37.203995 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.203997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.204302 (+ 305us) server_negotiation.cc:407] Connection header received
0810 07:23:37.204899 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.204900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.204915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.204969 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:37.205682 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.206193 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.207222 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.207438 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.207525 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.208450 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:37.208452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:37.208452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:37.208458 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:37.208473 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:37.208477 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:37.208528 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.208621 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.208651 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:23:37.208925 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 10739
I20260810 07:23:37.209012 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.203884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:43755 (local address 127.14.164.3:42785)
0810 07:23:37.204124 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.204170 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.204780 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.205052 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.205058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.205071 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:37.205585 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.205590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.206309 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.206310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.207122 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.207124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.207538 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.207539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.207684 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.208296 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:37.208297 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:37.208328 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:37.208351 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:37.208527 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:37.208533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:37.208536 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.208608 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.208741 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260810 07:23:37.209049 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:37.209285 14992 external_mini_cluster.cc:1428] Setting key ff3f75bedf6bf8220993686088de79b1
I20260810 07:23:37.209616 10918 heartbeater.cc:347] Connected to a master server at 127.14.164.62:43755
I20260810 07:23:37.209739 10918 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:37.209975 10918 heartbeater.cc:511] Master 127.14.164.62:43755 requested a full tablet report, sending...
I20260810 07:23:37.210557 10248 ts_manager.cc:194] Registered new tserver with Master: e56fca4b4d5141dab0641b22dcbb342c (127.14.164.3:41433)
I20260810 07:23:37.211037 10248 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.3:42785
I20260810 07:23:37.222543 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:37.233493 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.224303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.14.164.62:43755)
0810 07:23:37.224396 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.224398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.224520 (+ 122us) server_negotiation.cc:407] Connection header received
0810 07:23:37.224551 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.224552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.224563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.224623 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:37.224844 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.225373 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.226282 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.226472 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.226540 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.233159 (+ 6619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:37.233160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:37.233161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:37.233167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:37.233184 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:37.233188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:37.233244 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.233322 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.233351 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:23:37.235802 10248 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:37.237538 10248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:37.244055 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.239868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44235 (local address 127.0.0.1:54830)
0810 07:23:37.240252 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.240263 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.240308 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.240544 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.240548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.240550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:37.240624 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.240626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.241678 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.241679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.243110 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.243112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.243767 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.243768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.243843 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.243856 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.243923 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.243975 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260810 07:23:37.244066 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.240161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.14.164.1:44235)
0810 07:23:37.240341 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.240347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.240356 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:37.240405 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.240406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.240421 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.240513 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:37.240711 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.241589 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.243201 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.243674 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.243767 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.243868 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.243934 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.243973 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260810 07:23:37.244398 10934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.239867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:33105 (local address 127.0.0.1:55062)
0810 07:23:37.240045 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.240091 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.240162 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.240903 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.240904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.240906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:37.240942 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.240943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.242237 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.242238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.243121 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.243125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.244028 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.244029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.244090 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.244094 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.244272 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.244317 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260810 07:23:37.244627 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.240080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.14.164.3:41433)
0810 07:23:37.240396 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.240400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.240409 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:37.240448 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.240450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.240472 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.240562 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:37.240789 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.241922 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.243140 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.243873 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.243946 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.244039 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.244504 (+ 465us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.244545 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":86,"threads_started":1}
I20260810 07:23:37.244694 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.239962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.14.164.2:33105)
0810 07:23:37.240711 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.240715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.240725 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:37.240786 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.240788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.240803 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.241016 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:37.241080 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.242108 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.243249 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.243937 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.244362 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.244539 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.244563 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.244599 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":91,"threads_started":1}
I20260810 07:23:37.244318 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.239868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:41433 (local address 127.0.0.1:42330)
0810 07:23:37.240271 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.240282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.240341 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.240559 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.240560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.240563 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:37.240648 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.240651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.242019 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.242020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.243068 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.243071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.244067 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.244068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.244121 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.244125 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.244174 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.244228 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":245,"thread_start_us":83,"threads_started":1}
I20260810 07:23:37.244915 10430 tablet_service.cc:1467] Processing CreateTablet for tablet 09d132b58dbd4d6ca10c1e2ca4b87517 (DEFAULT_TABLE table=test-table [id=3cb4d3cf278440d5b60bab8959ae6f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:37.245352 10430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09d132b58dbd4d6ca10c1e2ca4b87517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:37.245601 10642 tablet_service.cc:1467] Processing CreateTablet for tablet 09d132b58dbd4d6ca10c1e2ca4b87517 (DEFAULT_TABLE table=test-table [id=3cb4d3cf278440d5b60bab8959ae6f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:37.245604 10821 tablet_service.cc:1467] Processing CreateTablet for tablet 09d132b58dbd4d6ca10c1e2ca4b87517 (DEFAULT_TABLE table=test-table [id=3cb4d3cf278440d5b60bab8959ae6f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:37.245947 10821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09d132b58dbd4d6ca10c1e2ca4b87517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:37.245947 10642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09d132b58dbd4d6ca10c1e2ca4b87517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:37.249018 10940 tablet_bootstrap.cc:492] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: Bootstrap starting.
I20260810 07:23:37.249625 10941 tablet_bootstrap.cc:492] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Bootstrap starting.
I20260810 07:23:37.249871 10940 tablet_bootstrap.cc:654] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:37.250195 10940 log.cc:824] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:37.250236 10941 tablet_bootstrap.cc:654] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:37.250634 10941 log.cc:824] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:37.250964 10942 tablet_bootstrap.cc:492] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Bootstrap starting.
I20260810 07:23:37.251086 10940 tablet_bootstrap.cc:492] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: No bootstrap required, opened a new log
I20260810 07:23:37.251165 10940 ts_tablet_manager.cc:1397] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:37.251400 10941 tablet_bootstrap.cc:492] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: No bootstrap required, opened a new log
I20260810 07:23:37.251467 10941 ts_tablet_manager.cc:1397] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:37.251616 10942 tablet_bootstrap.cc:654] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:37.251959 10942 log.cc:824] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:37.252696 10942 tablet_bootstrap.cc:492] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: No bootstrap required, opened a new log
I20260810 07:23:37.252836 10942 ts_tablet_manager.cc:1397] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:37.253979 10940 raft_consensus.cc:348] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.254164 10940 raft_consensus.cc:374] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:37.254204 10940 raft_consensus.cc:729] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8efcb15cbcb441c6842123c6d71c625b, State: Initialized, Role: FOLLOWER
I20260810 07:23:37.254251 10941 raft_consensus.cc:348] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.254405 10941 raft_consensus.cc:374] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:37.254359 10940 consensus_queue.cc:260] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.254447 10941 raft_consensus.cc:729] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56fca4b4d5141dab0641b22dcbb342c, State: Initialized, Role: FOLLOWER
I20260810 07:23:37.254580 10941 consensus_queue.cc:260] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.254643 10940 ts_tablet_manager.cc:1428] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 07:23:37.254753 10538 heartbeater.cc:503] Master 127.14.164.62:43755 was elected leader, sending a full tablet report...
I20260810 07:23:37.254901 10918 heartbeater.cc:503] Master 127.14.164.62:43755 was elected leader, sending a full tablet report...
I20260810 07:23:37.255077 10942 raft_consensus.cc:348] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.255191 10941 ts_tablet_manager.cc:1428] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:23:37.255263 10942 raft_consensus.cc:374] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:37.255327 10942 raft_consensus.cc:729] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7744c2be3d5454383ab081981c14b57, State: Initialized, Role: FOLLOWER
I20260810 07:23:37.255443 10942 consensus_queue.cc:260] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.255702 10942 ts_tablet_manager.cc:1428] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:23:37.255827 10732 heartbeater.cc:503] Master 127.14.164.62:43755 was elected leader, sending a full tablet report...
I20260810 07:23:37.307914 10949 raft_consensus.cc:482] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:37.308104 10949 raft_consensus.cc:504] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.308529 10949 leader_election.cc:288] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7744c2be3d5454383ab081981c14b57 (127.14.164.2:33105), e56fca4b4d5141dab0641b22dcbb342c (127.14.164.3:41433)
I20260810 07:23:37.312778 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.308929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:50787 (local address 127.14.164.2:33105)
0810 07:23:37.308984 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.308987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.308998 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:23:37.309030 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.309031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.309042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.309131 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:37.309313 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.309843 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.311144 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.311888 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.312092 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.312102 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.312478 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.312564 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:23:37.312781 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.308771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:33105 (local address 127.14.164.1:50787)
0810 07:23:37.308868 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.308880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.308932 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.309153 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.309154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.309156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:37.309235 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.309237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.309950 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.309952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.310910 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.310918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.312095 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.312096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.312275 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.312278 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.312458 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.312546 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 07:23:37.312996 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.308930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:41013 (local address 127.14.164.3:41433)
0810 07:23:37.308992 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.308994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.309121 (+ 127us) server_negotiation.cc:407] Connection header received
0810 07:23:37.309186 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.309187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.309202 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.309264 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:37.309447 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.309935 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.311225 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.312003 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.312285 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.312317 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.312731 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.312846 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:23:37.313030 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.308780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:41433 (local address 127.14.164.1:41013)
0810 07:23:37.309056 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.309063 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:37.309126 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:37.309282 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:37.309283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:37.309285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:37.309359 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.309361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.310023 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.310024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.311024 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:37.311027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.312275 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.312276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:37.312475 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.312479 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:37.312701 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 07:23:37.312808 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":78,"thread_start_us":124,"threads_started":1}
I20260810 07:23:37.313663 10672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d132b58dbd4d6ca10c1e2ca4b87517" candidate_uuid: "8efcb15cbcb441c6842123c6d71c625b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7744c2be3d5454383ab081981c14b57" is_pre_election: true
I20260810 07:23:37.313931 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d132b58dbd4d6ca10c1e2ca4b87517" candidate_uuid: "8efcb15cbcb441c6842123c6d71c625b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56fca4b4d5141dab0641b22dcbb342c" is_pre_election: true
I20260810 07:23:37.314064 10672 raft_consensus.cc:2454] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8efcb15cbcb441c6842123c6d71c625b in term 0.
I20260810 07:23:37.314257 10847 raft_consensus.cc:2454] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8efcb15cbcb441c6842123c6d71c625b in term 0.
I20260810 07:23:37.314715 10364 leader_election.cc:302] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8efcb15cbcb441c6842123c6d71c625b, c7744c2be3d5454383ab081981c14b57; no voters:
I20260810 07:23:37.315030 10949 raft_consensus.cc:2781] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:37.315220 10949 raft_consensus.cc:482] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:37.315397 10949 raft_consensus.cc:3037] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:37.317462 10949 raft_consensus.cc:504] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.317972 10949 leader_election.cc:288] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [CANDIDATE]: Term 1 election: Requested vote from peers c7744c2be3d5454383ab081981c14b57 (127.14.164.2:33105), e56fca4b4d5141dab0641b22dcbb342c (127.14.164.3:41433)
I20260810 07:23:37.318429 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d132b58dbd4d6ca10c1e2ca4b87517" candidate_uuid: "8efcb15cbcb441c6842123c6d71c625b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56fca4b4d5141dab0641b22dcbb342c"
I20260810 07:23:37.318744 10847 raft_consensus.cc:3037] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:23:37.319499 10735 tablet.cc:2369] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:37.321029 10672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d132b58dbd4d6ca10c1e2ca4b87517" candidate_uuid: "8efcb15cbcb441c6842123c6d71c625b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7744c2be3d5454383ab081981c14b57"
I20260810 07:23:37.321210 10672 raft_consensus.cc:3037] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:37.321584 10847 raft_consensus.cc:2454] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8efcb15cbcb441c6842123c6d71c625b in term 1.
I20260810 07:23:37.322150 10362 leader_election.cc:302] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8efcb15cbcb441c6842123c6d71c625b, e56fca4b4d5141dab0641b22dcbb342c; no voters:
I20260810 07:23:37.322465 10949 raft_consensus.cc:2781] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:37.323045 10949 raft_consensus.cc:686] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 1 LEADER]: Becoming Leader. State: Replica: 8efcb15cbcb441c6842123c6d71c625b, State: Running, Role: LEADER
I20260810 07:23:37.323298 10949 consensus_queue.cc:237] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } }
I20260810 07:23:37.323697 10672 raft_consensus.cc:2454] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8efcb15cbcb441c6842123c6d71c625b in term 1.
I20260810 07:23:37.326273 10248 catalog_manager.cc:5495] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8efcb15cbcb441c6842123c6d71c625b (127.14.164.1). New cstate: current_term: 1 leader_uuid: "8efcb15cbcb441c6842123c6d71c625b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8efcb15cbcb441c6842123c6d71c625b" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 44235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 } health_report { overall_health: UNKNOWN } } }
I20260810 07:23:37.346207 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.340792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.14.164.1:44235)
0810 07:23:37.340891 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:37.340902 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:37.340956 (+ 54us) server_negotiation.cc:407] Connection header received
0810 07:23:37.341206 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:37.341207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:37.341296 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:37.341518 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:37.341861 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.342725 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.344409 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:37.344691 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:37.344954 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:37.345332 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:37.345564 (+ 232us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:37.345848 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:37.345901 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 07:23:37.345935 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:23:37.350447 10672 raft_consensus.cc:1261] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 1 FOLLOWER]: Refusing update from remote peer 8efcb15cbcb441c6842123c6d71c625b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:37.350485 10847 raft_consensus.cc:1261] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 1 FOLLOWER]: Refusing update from remote peer 8efcb15cbcb441c6842123c6d71c625b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:37.351182 10957 consensus_queue.cc:1035] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56fca4b4d5141dab0641b22dcbb342c" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 41433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:37.351377 10949 consensus_queue.cc:1035] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7744c2be3d5454383ab081981c14b57" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:37.357191 10965 mvcc.cc:205] Tried to move back new op lower bound from 7316875744662786048 to 7316875744558858240. Current Snapshot: MvccSnapshot[applied={T|T < 7316875744662786048}]
I20260810 07:23:37.358619 10966 mvcc.cc:205] Tried to move back new op lower bound from 7316875744662786048 to 7316875744558858240. Current Snapshot: MvccSnapshot[applied={T|T < 7316875744662786048 or (T in {7316875744662786048})}]
I20260810 07:23:37.363790 10248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:37.363991 10248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:37.366079 10248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 867ac9e9d2934dbb8fabd244c3341feb: Sending DeleteTablet for 3 replicas of tablet 09d132b58dbd4d6ca10c1e2ca4b87517
I20260810 07:23:37.367347 10430 tablet_service.cc:1514] Processing DeleteTablet for tablet 09d132b58dbd4d6ca10c1e2ca4b87517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:37 UTC) from {username='slave'} at 127.0.0.1:54830
I20260810 07:23:37.367321 10821 tablet_service.cc:1514] Processing DeleteTablet for tablet 09d132b58dbd4d6ca10c1e2ca4b87517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:37 UTC) from {username='slave'} at 127.0.0.1:42330
I20260810 07:23:37.367347 10642 tablet_service.cc:1514] Processing DeleteTablet for tablet 09d132b58dbd4d6ca10c1e2ca4b87517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:37 UTC) from {username='slave'} at 127.0.0.1:55062
I20260810 07:23:37.367903 10975 tablet_replica.cc:330] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b: stopping tablet replica
I20260810 07:23:37.368173 10975 raft_consensus.cc:2229] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:23:37.368566 10975 raft_consensus.cc:2258] T 09d132b58dbd4d6ca10c1e2ca4b87517 P 8efcb15cbcb441c6842123c6d71c625b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:37.368986 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 10341
I20260810 07:23:37.397481 10976 tablet_replica.cc:330] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: stopping tablet replica
I20260810 07:23:37.398025 10976 raft_consensus.cc:2229] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:37.398109 10974 tablet_replica.cc:330] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: stopping tablet replica
I20260810 07:23:37.398406 10974 raft_consensus.cc:2229] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:37.398496 10976 raft_consensus.cc:2258] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:37.398634 10974 raft_consensus.cc:2258] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:37.400223 10974 ts_tablet_manager.cc:1905] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:37.400221 10976 ts_tablet_manager.cc:1905] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:37.404832 10976 ts_tablet_manager.cc:1918] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:37.404832 10974 ts_tablet_manager.cc:1918] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:37.405128 10974 log.cc:1196] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/09d132b58dbd4d6ca10c1e2ca4b87517
I20260810 07:23:37.405097 10976 log.cc:1196] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/09d132b58dbd4d6ca10c1e2ca4b87517
I20260810 07:23:37.406339 10974 ts_tablet_manager.cc:1939] T 09d132b58dbd4d6ca10c1e2ca4b87517 P e56fca4b4d5141dab0641b22dcbb342c: Deleting consensus metadata
I20260810 07:23:37.406339 10976 ts_tablet_manager.cc:1939] T 09d132b58dbd4d6ca10c1e2ca4b87517 P c7744c2be3d5454383ab081981c14b57: Deleting consensus metadata
I20260810 07:23:37.407866 10229 catalog_manager.cc:4841] TS e56fca4b4d5141dab0641b22dcbb342c (127.14.164.3:41433): tablet 09d132b58dbd4d6ca10c1e2ca4b87517 (table test-table [id=3cb4d3cf278440d5b60bab8959ae6f82]) successfully deleted
I20260810 07:23:37.407872 10231 catalog_manager.cc:4841] TS c7744c2be3d5454383ab081981c14b57 (127.14.164.2:33105): tablet 09d132b58dbd4d6ca10c1e2ca4b87517 (table test-table [id=3cb4d3cf278440d5b60bab8959ae6f82]) successfully deleted
W20260810 07:23:37.409179 10232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:44235 (error 108)
I20260810 07:23:37.410259 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.409898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44235 (local address 127.0.0.1:54844)
0810 07:23:37.410086 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.410164 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:44235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 07:23:37.410281 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 10543
W20260810 07:23:37.428884 10232 catalog_manager.cc:4584] TS 8efcb15cbcb441c6842123c6d71c625b (127.14.164.1:44235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09d132b58dbd4d6ca10c1e2ca4b87517: Network error: Client connection negotiation failed: client connection to 127.14.164.1:44235: connect: Connection refused (error 111)
I20260810 07:23:37.442044 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 10739
I20260810 07:23:37.482846 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.482428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44235 (local address 127.0.0.1:54846)
0810 07:23:37.482585 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.482717 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:44235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 07:23:37.483970 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:37.483731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:44235 (local address 127.0.0.1:54862)
0810 07:23:37.483813 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:37.483878 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:44235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:23:37.489208 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 10209
2026-08-10T07:23:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (943 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:37 dist-test-slave-x5fp krb5kdc[10980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:37 dist-test-slave-x5fp krb5kdc[10980](info): set up 2 sockets
Aug 10 07:23:37 dist-test-slave-x5fp krb5kdc[10980](info): commencing operation
krb5kdc: starting...
W20260810 07:23:38.849877 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.005s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:38 dist-test-slave-x5fp krb5kdc[10980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:23:38.992625 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39583
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:39.082957 10997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:39.083176 10997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:39.083199 10997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:39.084389 10997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:39.084437 10997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:39.084452 10997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:39.084465 10997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:39.084478 10997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:39.086098 10997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:39.086369 10997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:39.086642 10997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:39.091992 11006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:39.092101 11007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:39.092051 11009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:39.092288 10997 server_base.cc:1034] running on GCE node
I20260810 07:23:39.092608 10997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:39.093084 10997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:39.094306 10997 hybrid_clock.cc:648] HybridClock initialized: now 1786346619094271 us; error 43 us; skew 500 ppm
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:39.097182 10997 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:23:39.097630 10997 webserver.cc:460] Webserver started at http://127.14.164.62:40473/ using document root <none> and password file <none>
I20260810 07:23:39.097884 10997 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:39.097932 10997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:39.098096 10997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:39.099256 10997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "54d671fb14e94f2e8c1d41519106bce0"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "28952aa05e39e0b418d02a119d6ac4fd"
server_key_iv: "00d01f7967b3a8ebff792bca750e6ceb"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.099670 10997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "54d671fb14e94f2e8c1d41519106bce0"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "28952aa05e39e0b418d02a119d6ac4fd"
server_key_iv: "00d01f7967b3a8ebff792bca750e6ceb"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.101269 10997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:39.102182 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.102631 10997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:39.102727 10997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "54d671fb14e94f2e8c1d41519106bce0"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "28952aa05e39e0b418d02a119d6ac4fd"
server_key_iv: "00d01f7967b3a8ebff792bca750e6ceb"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.102821 10997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:39.115458 10997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:39.117738 10997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:39.117962 10997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:39.122273 10997 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:39583
I20260810 07:23:39.122434 11090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:39583 every 8 connection(s)
I20260810 07:23:39.122721 10997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:39.123760 11092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:39.126307 11092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: Bootstrap starting.
I20260810 07:23:39.126875 11092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:39.127130 11092 log.cc:824] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:39.127854 11092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: No bootstrap required, opened a new log
I20260810 07:23:39.129657 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 10997
I20260810 07:23:39.129604 11092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d671fb14e94f2e8c1d41519106bce0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39583 } }
I20260810 07:23:39.129765 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:39.129805 11092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:39.129856 11092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d671fb14e94f2e8c1d41519106bce0, State: Initialized, Role: FOLLOWER
I20260810 07:23:39.130048 14992 external_mini_cluster.cc:1428] Setting key 02bf008a7413ca9e32fa003bb740eed7
I20260810 07:23:39.130033 11092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d671fb14e94f2e8c1d41519106bce0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39583 } }
I20260810 07:23:39.130125 11092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:39.130167 11092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:39.130218 11092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:39.131520 11092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d671fb14e94f2e8c1d41519106bce0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39583 } }
I20260810 07:23:39.131624 11092 leader_election.cc:302] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d671fb14e94f2e8c1d41519106bce0; no voters:
I20260810 07:23:39.131814 11092 leader_election.cc:288] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:39.131876 11100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:39.132028 11100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [term 1 LEADER]: Becoming Leader. State: Replica: 54d671fb14e94f2e8c1d41519106bce0, State: Running, Role: LEADER
I20260810 07:23:39.132118 11092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:39.132148 11100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d671fb14e94f2e8c1d41519106bce0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39583 } }
I20260810 07:23:39.132551 11100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d671fb14e94f2e8c1d41519106bce0. Latest consensus state: current_term: 1 leader_uuid: "54d671fb14e94f2e8c1d41519106bce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d671fb14e94f2e8c1d41519106bce0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39583 } } }
I20260810 07:23:39.132627 11100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:39.132809 11101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d671fb14e94f2e8c1d41519106bce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d671fb14e94f2e8c1d41519106bce0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39583 } } }
I20260810 07:23:39.132995 11101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0 [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:39.134303 11123 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:39.134601 11123 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:39.134677 11109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:39.134843 11109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:39.136479 11109 catalog_manager.cc:1347] Generated new cluster ID: c5203e138c1c4395b063c33eb326719c
I20260810 07:23:39.136539 11109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:39.141714 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.130614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.14.164.62:39583)
0810 07:23:39.130859 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.130878 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.130934 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:23:39.131105 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.131115 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.131184 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.131345 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:39.132296 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.133864 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.134911 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.135394 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.135524 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.138142 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:39.138146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:39.138146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:39.138159 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:39.139922 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.140329 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.140332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.140333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.140444 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.140667 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.140668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.140668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.140997 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:39.141180 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.141282 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.141549 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":110,"threads_started":1}
I20260810 07:23:39.142524 11109 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:39.143024 11109 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:39.151610 11109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: Generated new TSK 0
I20260810 07:23:39.151842 11109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:23:39.195705 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39583
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:39.295535 11130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:39.295758 11130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:39.295783 11130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:39.297000 11130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:39.297035 11130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:39.297061 11130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:39.298545 11130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:39.298755 11130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:39.298961 11130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:39.304363 11136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:39.304463 11130 server_base.cc:1034] running on GCE node
W20260810 07:23:39.304347 11138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:39.304371 11140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:39.304833 11130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:39.305142 11130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:39.306329 11130 hybrid_clock.cc:648] HybridClock initialized: now 1786346619306293 us; error 51 us; skew 500 ppm
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:39.309361 11130 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:23:39.309890 11130 webserver.cc:460] Webserver started at http://127.14.164.1:35251/ using document root <none> and password file <none>
I20260810 07:23:39.310184 11130 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:39.310298 11130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:39.310470 11130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:39.311656 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "456703dbac0e447aa7fd42a3082c354c"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "dbcf01158f020241aadf6aec1aa38f20"
server_key_iv: "407c737403a852d6d9d6c24db27f7b53"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.312229 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "456703dbac0e447aa7fd42a3082c354c"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "dbcf01158f020241aadf6aec1aa38f20"
server_key_iv: "407c737403a852d6d9d6c24db27f7b53"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.313872 11130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 07:23:39.314706 11154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.314911 11130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:39.315027 11130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "456703dbac0e447aa7fd42a3082c354c"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "dbcf01158f020241aadf6aec1aa38f20"
server_key_iv: "407c737403a852d6d9d6c24db27f7b53"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.315109 11130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:39.333482 11130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:39.335837 11130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:39.335955 11130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:39.336144 11130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:39.336483 11130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:39.336519 11130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.336542 11130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:39.336589 11130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.342995 11130 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:33291
I20260810 07:23:39.343080 11315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:33291 every 8 connection(s)
I20260810 07:23:39.343488 11130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:39.353650 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 11130
I20260810 07:23:39.353776 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:39.354029 14992 external_mini_cluster.cc:1428] Setting key f1e52b3fa528286b80f540c63089a50a
I20260810 07:23:39.354436 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.344382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:54677 (local address 127.14.164.62:39583)
0810 07:23:39.344466 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.344468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.344683 (+ 215us) server_negotiation.cc:407] Connection header received
0810 07:23:39.345256 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.345258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.345272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.345349 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:39.346766 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.347274 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.348328 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.348531 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.348598 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.350969 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:39.350970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:39.350971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:39.350977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:39.352622 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.353183 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.353184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.353185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.353252 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.353600 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.353602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.353602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.353789 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:39.353871 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.354240 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.354340 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:23:39.354650 11318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.344319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39583 (local address 127.14.164.1:54677)
0810 07:23:39.344544 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.344589 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.345166 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.345410 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.345413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.346278 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:39.346655 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.346662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.347382 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.347383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.348216 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.348219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.348618 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.348619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.348778 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.349400 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:39.349408 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:39.350865 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:39.352816 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:39.352824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:39.352826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:39.353089 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:39.353356 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:39.353357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:39.353358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:39.353481 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:39.353882 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:39.353891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:39.354039 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.354161 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.354381 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260810 07:23:39.355182 11316 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39583
I20260810 07:23:39.355285 11316 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:39.355504 11316 heartbeater.cc:511] Master 127.14.164.62:39583 requested a full tablet report, sending...
I20260810 07:23:39.356040 11037 ts_manager.cc:194] Registered new tserver with Master: 456703dbac0e447aa7fd42a3082c354c (127.14.164.1:33291)
I20260810 07:23:39.356619 11037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:54677
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:23:39.387518 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39583
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:39.473342 11321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:39.473577 11321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:39.473605 11321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:39.474828 11321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:39.474879 11321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:39.474948 11321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:39.476956 11321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:39.477236 11321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:39.477526 11321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:39.482904 11327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:39.482904 11328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:39.482967 11330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:39.483089 11321 server_base.cc:1034] running on GCE node
I20260810 07:23:39.483455 11321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:39.483717 11321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:39.484910 11321 hybrid_clock.cc:648] HybridClock initialized: now 1786346619484897 us; error 32 us; skew 500 ppm
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:39.487674 11321 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:23:39.488360 11321 webserver.cc:460] Webserver started at http://127.14.164.2:41427/ using document root <none> and password file <none>
I20260810 07:23:39.488641 11321 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:39.488752 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:39.488917 11321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:39.490267 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "6ef0da96a72c4680bea1f7dd511ddd61"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "48cb8d75e64df434ef8767a7da7ec3ad"
server_key_iv: "ef3012245231a01ff9a60ceef62d672d"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.490743 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "6ef0da96a72c4680bea1f7dd511ddd61"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "48cb8d75e64df434ef8767a7da7ec3ad"
server_key_iv: "ef3012245231a01ff9a60ceef62d672d"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.492367 11321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:39.493516 11341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.493810 11321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:39.493925 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "6ef0da96a72c4680bea1f7dd511ddd61"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "48cb8d75e64df434ef8767a7da7ec3ad"
server_key_iv: "ef3012245231a01ff9a60ceef62d672d"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.494009 11321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:39.517519 11321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:39.520316 11321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:39.520506 11321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:39.520846 11321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:39.521242 11321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:39.521283 11321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.521310 11321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:39.521324 11321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.528270 11321 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:39313
I20260810 07:23:39.528318 11503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:39313 every 8 connection(s)
I20260810 07:23:39.528688 11321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:39.534384 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 11321
I20260810 07:23:39.534523 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:39.534736 14992 external_mini_cluster.cc:1428] Setting key 62e1a75fcc67de1ec5ad4d8df054e987
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:39.539857 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.529527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:44379 (local address 127.14.164.62:39583)
0810 07:23:39.529580 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.529587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.529794 (+ 207us) server_negotiation.cc:407] Connection header received
0810 07:23:39.530375 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.530377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.530389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.530457 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:39.531551 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.532105 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.533215 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.533499 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.533560 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.536067 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:39.536069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:39.536069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:39.536078 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:39.537896 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.538518 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.538518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.538519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.538578 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.538929 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.538931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.538931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.539078 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:39.539152 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.539588 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.539701 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:23:39.539871 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.529495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39583 (local address 127.14.164.2:44379)
0810 07:23:39.529673 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.529717 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.530252 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.530536 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.530539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.531169 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:39.531397 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.531403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.532227 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.532228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.533125 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.533127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.533581 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.533581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.533693 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.534283 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:39.534295 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:39.535975 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:39.537998 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:39.538008 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:39.538011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:39.538429 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:39.538655 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:39.538656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:39.538657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:39.538830 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:39.539166 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:39.539173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:39.539369 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.539513 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.539661 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260810 07:23:39.540470 11504 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39583
I20260810 07:23:39.540596 11504 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:39.540864 11504 heartbeater.cc:511] Master 127.14.164.62:39583 requested a full tablet report, sending...
I20260810 07:23:39.541388 11037 ts_manager.cc:194] Registered new tserver with Master: 6ef0da96a72c4680bea1f7dd511ddd61 (127.14.164.2:39313)
I20260810 07:23:39.541867 11037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:44379
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:23:39.568807 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39583
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:39.658460 11511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:39.658712 11511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:39.658740 11511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:39.659925 11511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:39.659965 11511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:39.659991 11511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:39.661558 11511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:39.661778 11511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:39.661972 11511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:39.667430 11519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:39.667451 11524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:39.667625 11511 server_base.cc:1034] running on GCE node
W20260810 07:23:39.667412 11518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:39.667979 11511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:39.668212 11511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:39.669396 11511 hybrid_clock.cc:648] HybridClock initialized: now 1786346619669379 us; error 33 us; skew 500 ppm
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:39.672488 11511 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:23:39.673064 11511 webserver.cc:460] Webserver started at http://127.14.164.3:41877/ using document root <none> and password file <none>
I20260810 07:23:39.673316 11511 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:39.673386 11511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:39.673537 11511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:39.674640 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "1ef48122fac9499caa7150f5a811f04e"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "9bc6b70d7ea189140eccd825478979e4"
server_key_iv: "79576c7a5216ffb7d39572bdad9a8877"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.675019 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "1ef48122fac9499caa7150f5a811f04e"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "9bc6b70d7ea189140eccd825478979e4"
server_key_iv: "79576c7a5216ffb7d39572bdad9a8877"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.676888 11511 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:23:39.677732 11537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.677964 11511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:39.678074 11511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "1ef48122fac9499caa7150f5a811f04e"
format_stamp: "Formatted at 2026-08-10 07:23:39 on dist-test-slave-x5fp"
server_key: "9bc6b70d7ea189140eccd825478979e4"
server_key_iv: "79576c7a5216ffb7d39572bdad9a8877"
server_key_version: "encryptionkey@0"
I20260810 07:23:39.678141 11511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:39.691452 11511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:39.693881 11511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:39.694002 11511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:39.694202 11511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:39.694530 11511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:39.694566 11511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.694612 11511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:39.694639 11511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:39.701364 11511 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:42513
I20260810 07:23:39.701422 11690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:42513 every 8 connection(s)
I20260810 07:23:39.701753 11511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:39.705678 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 11511
I20260810 07:23:39.705804 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:39.706002 14992 external_mini_cluster.cc:1428] Setting key b1ec9d27548ba33e24e6f20f6da353ce
Aug 10 07:23:39 dist-test-slave-x5fp krb5kdc[10980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346619, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:39.711750 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.702513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:51767 (local address 127.14.164.62:39583)
0810 07:23:39.702580 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.702582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.702811 (+ 229us) server_negotiation.cc:407] Connection header received
0810 07:23:39.703405 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.703406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.703420 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.703482 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:39.704489 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.705089 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.706319 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.706640 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.706720 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.709104 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:39.709105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:39.709106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:39.709113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:39.710559 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.710940 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.710941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.710942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.710996 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.711219 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.711220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.711220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.711324 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:39.711375 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.711619 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.711684 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:23:39.712013 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.702469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39583 (local address 127.14.164.3:51767)
0810 07:23:39.702669 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.702714 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.703284 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.703556 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.703560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.704171 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:39.704382 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.704387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.705201 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.705202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.706217 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.706220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.706745 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.706746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.706970 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.707482 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:39.707491 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:39.709018 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:39.710632 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:39.710638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:39.710640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:39.710872 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:39.711057 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:39.711057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:39.711058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:39.711163 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:39.711379 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:39.711384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:39.711505 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.711590 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.711715 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260810 07:23:39.712574 11691 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39583
I20260810 07:23:39.712692 11691 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:39.712940 11691 heartbeater.cc:511] Master 127.14.164.62:39583 requested a full tablet report, sending...
I20260810 07:23:39.713441 11037 ts_manager.cc:194] Registered new tserver with Master: 1ef48122fac9499caa7150f5a811f04e (127.14.164.3:42513)
I20260810 07:23:39.713795 11037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:51767
I20260810 07:23:39.718286 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:39.726930 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.720041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.14.164.62:39583)
0810 07:23:39.720127 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.720129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.720241 (+ 112us) server_negotiation.cc:407] Connection header received
0810 07:23:39.720284 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.720284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.720293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.720347 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:39.720994 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.721488 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.722277 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.722466 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.722515 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.723998 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:39.724000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:39.724000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:39.724008 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:39.725556 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.726091 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.726092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.726093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.726159 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:39.726395 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:39.726396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:39.726396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:39.726542 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:39.726626 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.726800 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.726855 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:23:39.728641 11037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:39.730304 11037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:39.737514 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.732985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.14.164.3:42513)
0810 07:23:39.733326 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.733332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.733456 (+ 124us) server_negotiation.cc:407] Connection header received
0810 07:23:39.733611 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.733613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.733629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.733703 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:39.733946 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.735043 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.736419 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.737058 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.737145 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.737205 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.737375 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.737419 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":87,"threads_started":1}
I20260810 07:23:39.737514 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.732946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:42513 (local address 127.0.0.1:57544)
0810 07:23:39.733372 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.733382 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.733505 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.733814 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.733815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.733817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:39.733879 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.733881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.735158 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.735159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.736320 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.736323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.737147 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.737148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.737233 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.737246 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.737354 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.737424 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260810 07:23:39.737522 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.733143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:33291 (local address 127.0.0.1:57636)
0810 07:23:39.733406 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.733417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.733482 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.733722 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.733726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.733728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:39.733815 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.733817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.735178 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.735179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.736211 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.736214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.737231 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.737232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.737304 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.737309 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.737376 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.737429 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260810 07:23:39.737895 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.733196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57636 (local address 127.14.164.1:33291)
0810 07:23:39.733387 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.733392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.733567 (+ 175us) server_negotiation.cc:407] Connection header received
0810 07:23:39.733613 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.733615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.733632 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.733718 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:39.733897 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.735056 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.736300 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.737059 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.737132 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.737191 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.737565 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.737754 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260810 07:23:39.738215 11710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.732958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.14.164.2:39313)
0810 07:23:39.733173 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.733179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.733426 (+ 247us) server_negotiation.cc:407] Connection header received
0810 07:23:39.733607 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.733610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.733628 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.733688 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:39.734110 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.734860 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.736567 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.737347 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.737410 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.737462 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.737983 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.738036 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":105,"threads_started":1}
I20260810 07:23:39.738374 11596 tablet_service.cc:1467] Processing CreateTablet for tablet f84e0751d9eb4b108dfefbd0db987183 (DEFAULT_TABLE table=test-table [id=01d7222c4783432f816d649fa462cdba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:39.738775 11596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84e0751d9eb4b108dfefbd0db987183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:39.738713 11217 tablet_service.cc:1467] Processing CreateTablet for tablet f84e0751d9eb4b108dfefbd0db987183 (DEFAULT_TABLE table=test-table [id=01d7222c4783432f816d649fa462cdba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:39.739022 11217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84e0751d9eb4b108dfefbd0db987183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:39.741722 11721 tablet_bootstrap.cc:492] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Bootstrap starting.
I20260810 07:23:39.741783 11722 tablet_bootstrap.cc:492] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: Bootstrap starting.
I20260810 07:23:39.742018 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.732823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:39313 (local address 127.0.0.1:59286)
0810 07:23:39.733284 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.733332 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.733476 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.733996 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.733997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.733998 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:39.734035 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.734036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.734978 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.734979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.736475 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.736477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.737496 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.737497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.737566 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.737570 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.737940 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.738074 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":66,"thread_start_us":55,"threads_started":1}
I20260810 07:23:39.742513 11721 tablet_bootstrap.cc:654] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:39.742624 11722 tablet_bootstrap.cc:654] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:39.742893 11414 tablet_service.cc:1467] Processing CreateTablet for tablet f84e0751d9eb4b108dfefbd0db987183 (DEFAULT_TABLE table=test-table [id=01d7222c4783432f816d649fa462cdba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:39.743197 11722 log.cc:824] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:39.743196 11721 log.cc:824] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:39.743304 11414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84e0751d9eb4b108dfefbd0db987183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:39.743937 11722 tablet_bootstrap.cc:492] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: No bootstrap required, opened a new log
I20260810 07:23:39.744027 11722 ts_tablet_manager.cc:1397] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:39.745769 11722 raft_consensus.cc:348] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.745946 11722 raft_consensus.cc:374] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:39.745996 11722 raft_consensus.cc:729] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456703dbac0e447aa7fd42a3082c354c, State: Initialized, Role: FOLLOWER
I20260810 07:23:39.746146 11722 consensus_queue.cc:260] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.746429 11726 tablet_bootstrap.cc:492] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Bootstrap starting.
I20260810 07:23:39.746505 11722 ts_tablet_manager.cc:1428] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:23:39.746512 11316 heartbeater.cc:503] Master 127.14.164.62:39583 was elected leader, sending a full tablet report...
I20260810 07:23:39.747186 11721 tablet_bootstrap.cc:492] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: No bootstrap required, opened a new log
I20260810 07:23:39.747283 11721 ts_tablet_manager.cc:1397] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260810 07:23:39.747283 11726 tablet_bootstrap.cc:654] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:39.747934 11726 log.cc:824] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:39.748700 11726 tablet_bootstrap.cc:492] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: No bootstrap required, opened a new log
I20260810 07:23:39.748816 11726 ts_tablet_manager.cc:1397] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:39.749338 11721 raft_consensus.cc:348] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.749526 11721 raft_consensus.cc:374] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:39.749552 11721 raft_consensus.cc:729] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef48122fac9499caa7150f5a811f04e, State: Initialized, Role: FOLLOWER
I20260810 07:23:39.749696 11721 consensus_queue.cc:260] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.750000 11721 ts_tablet_manager.cc:1428] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:39.750038 11691 heartbeater.cc:503] Master 127.14.164.62:39583 was elected leader, sending a full tablet report...
I20260810 07:23:39.751045 11726 raft_consensus.cc:348] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.751227 11726 raft_consensus.cc:374] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:39.751281 11726 raft_consensus.cc:729] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef0da96a72c4680bea1f7dd511ddd61, State: Initialized, Role: FOLLOWER
I20260810 07:23:39.751405 11726 consensus_queue.cc:260] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.751699 11726 ts_tablet_manager.cc:1428] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:39.751708 11504 heartbeater.cc:503] Master 127.14.164.62:39583 was elected leader, sending a full tablet report...
W20260810 07:23:39.779441 11505 tablet.cc:2369] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:23:39.844338 11317 tablet.cc:2369] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:39.947657 11730 raft_consensus.cc:482] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:39.947829 11730 raft_consensus.cc:504] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.948336 11730 leader_election.cc:288] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ef0da96a72c4680bea1f7dd511ddd61 (127.14.164.2:39313), 456703dbac0e447aa7fd42a3082c354c (127.14.164.1:33291)
W20260810 07:23:39.952337 11693 tablet.cc:2369] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:39.953152 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.948708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:34555 (local address 127.14.164.1:33291)
0810 07:23:39.948823 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.948826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.949022 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:23:39.949058 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.949060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.949071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.949135 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:39.949404 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.950181 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.951975 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.952636 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.952797 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.952812 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.953021 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.953071 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 07:23:39.953152 11710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.948716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:54909 (local address 127.14.164.2:39313)
0810 07:23:39.948803 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.948806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.948819 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:23:39.948912 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.948915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.949009 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.949110 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:39.949408 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.950166 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.951974 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.952687 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.952825 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.952835 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.953021 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.953068 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:23:39.953162 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.948607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:39313 (local address 127.14.164.3:54909)
0810 07:23:39.948672 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.948685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.948769 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.949176 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.949177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.949180 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:39.949304 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.949306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.950306 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.950307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.951815 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.951820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.952820 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.952821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.952918 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.952926 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.952996 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.953054 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":104}
I20260810 07:23:39.953179 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.948607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:33291 (local address 127.14.164.3:34555)
0810 07:23:39.948883 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:39.948892 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:39.948964 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:39.949176 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:39.949177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:39.949180 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:39.949323 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.949324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.950305 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.950307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.951830 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:39.951832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.952815 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.952817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:39.952918 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.952926 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:23:39.952996 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:23:39.953051 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":183,"thread_start_us":162,"threads_started":1}
I20260810 07:23:39.953542 11442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84e0751d9eb4b108dfefbd0db987183" candidate_uuid: "1ef48122fac9499caa7150f5a811f04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" is_pre_election: true
I20260810 07:23:39.953534 11241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84e0751d9eb4b108dfefbd0db987183" candidate_uuid: "1ef48122fac9499caa7150f5a811f04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456703dbac0e447aa7fd42a3082c354c" is_pre_election: true
I20260810 07:23:39.953737 11241 raft_consensus.cc:2454] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ef48122fac9499caa7150f5a811f04e in term 0.
I20260810 07:23:39.953749 11442 raft_consensus.cc:2454] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ef48122fac9499caa7150f5a811f04e in term 0.
I20260810 07:23:39.954008 11543 leader_election.cc:302] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef48122fac9499caa7150f5a811f04e, 456703dbac0e447aa7fd42a3082c354c; no voters:
I20260810 07:23:39.954216 11730 raft_consensus.cc:2781] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:39.954315 11730 raft_consensus.cc:482] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:39.954355 11730 raft_consensus.cc:3037] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:39.955183 11730 raft_consensus.cc:504] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.955363 11730 leader_election.cc:288] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [CANDIDATE]: Term 1 election: Requested vote from peers 6ef0da96a72c4680bea1f7dd511ddd61 (127.14.164.2:39313), 456703dbac0e447aa7fd42a3082c354c (127.14.164.1:33291)
I20260810 07:23:39.955555 11442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84e0751d9eb4b108dfefbd0db987183" candidate_uuid: "1ef48122fac9499caa7150f5a811f04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef0da96a72c4680bea1f7dd511ddd61"
I20260810 07:23:39.955667 11442 raft_consensus.cc:3037] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:39.956852 11442 raft_consensus.cc:2454] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ef48122fac9499caa7150f5a811f04e in term 1.
I20260810 07:23:39.957079 11541 leader_election.cc:302] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef48122fac9499caa7150f5a811f04e, 6ef0da96a72c4680bea1f7dd511ddd61; no voters:
I20260810 07:23:39.957219 11730 raft_consensus.cc:2781] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:39.957468 11730 raft_consensus.cc:686] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 1 LEADER]: Becoming Leader. State: Replica: 1ef48122fac9499caa7150f5a811f04e, State: Running, Role: LEADER
I20260810 07:23:39.957602 11730 consensus_queue.cc:237] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } }
I20260810 07:23:39.958683 11037 catalog_manager.cc:5495] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ef48122fac9499caa7150f5a811f04e (127.14.164.3). New cstate: current_term: 1 leader_uuid: "1ef48122fac9499caa7150f5a811f04e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ef48122fac9499caa7150f5a811f04e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 42513 } health_report { overall_health: HEALTHY } } }
I20260810 07:23:39.959473 11241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84e0751d9eb4b108dfefbd0db987183" candidate_uuid: "1ef48122fac9499caa7150f5a811f04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456703dbac0e447aa7fd42a3082c354c"
I20260810 07:23:39.959583 11241 raft_consensus.cc:3037] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:39.960548 11241 raft_consensus.cc:2454] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ef48122fac9499caa7150f5a811f04e in term 1.
I20260810 07:23:39.966289 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:39.963721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57552 (local address 127.14.164.3:42513)
0810 07:23:39.963769 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:39.963772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:39.963782 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:39.963842 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:39.963843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:39.963864 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:39.963957 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:39.964116 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.964602 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.965431 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:39.965642 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:39.965730 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:39.965911 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:39.966013 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:39.966126 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:39.966157 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 07:23:39.966170 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:23:39.968195 11442 raft_consensus.cc:1261] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 1 FOLLOWER]: Refusing update from remote peer 1ef48122fac9499caa7150f5a811f04e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:39.968295 11241 raft_consensus.cc:1261] T f84e0751d9eb4b108dfefbd0db987183 P 456703dbac0e447aa7fd42a3082c354c [term 1 FOLLOWER]: Refusing update from remote peer 1ef48122fac9499caa7150f5a811f04e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:39.968633 11730 consensus_queue.cc:1035] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [LEADER]: Connected to new peer: Peer: permanent_uuid: "456703dbac0e447aa7fd42a3082c354c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 33291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:39.968834 11733 consensus_queue.cc:1035] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ef0da96a72c4680bea1f7dd511ddd61" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:39.971632 11736 mvcc.cc:205] Tried to move back new op lower bound from 7316875755386703872 to 7316875755346972672. Current Snapshot: MvccSnapshot[applied={T|T < 7316875755386703872}]
I20260810 07:23:39.972467 11738 mvcc.cc:205] Tried to move back new op lower bound from 7316875755386703872 to 7316875755346972672. Current Snapshot: MvccSnapshot[applied={T|T < 7316875755386703872}]
I20260810 07:23:39.972601 11737 mvcc.cc:205] Tried to move back new op lower bound from 7316875755386703872 to 7316875755346972672. Current Snapshot: MvccSnapshot[applied={T|T < 7316875755386703872}]
I20260810 07:23:39.974442 11038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:39.974583 11038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:39.975454 11038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54d671fb14e94f2e8c1d41519106bce0: Sending DeleteTablet for 3 replicas of tablet f84e0751d9eb4b108dfefbd0db987183
I20260810 07:23:39.975816 11414 tablet_service.cc:1514] Processing DeleteTablet for tablet f84e0751d9eb4b108dfefbd0db987183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:39 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:59286
I20260810 07:23:39.975999 11217 tablet_service.cc:1514] Processing DeleteTablet for tablet f84e0751d9eb4b108dfefbd0db987183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:39 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57636
I20260810 07:23:39.976238 11596 tablet_service.cc:1514] Processing DeleteTablet for tablet f84e0751d9eb4b108dfefbd0db987183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:39 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57544
I20260810 07:23:39.976267 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 11130
I20260810 07:23:39.991326 11751 tablet_replica.cc:330] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: stopping tablet replica
I20260810 07:23:39.991518 11751 raft_consensus.cc:2229] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:23:39.991679 11751 raft_consensus.cc:2258] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:39.992026 11748 tablet_replica.cc:330] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: stopping tablet replica
I20260810 07:23:39.992170 11748 raft_consensus.cc:2229] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:39.992303 11748 raft_consensus.cc:2258] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:39.992762 11751 ts_tablet_manager.cc:1905] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:39.994699 11751 ts_tablet_manager.cc:1918] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:39.994786 11751 log.cc:1196] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/f84e0751d9eb4b108dfefbd0db987183
I20260810 07:23:39.995097 11751 ts_tablet_manager.cc:1939] T f84e0751d9eb4b108dfefbd0db987183 P 1ef48122fac9499caa7150f5a811f04e: Deleting consensus metadata
I20260810 07:23:39.995409 11748 ts_tablet_manager.cc:1905] T f84e0751d9eb4b108dfefbd0db987183 P 6ef0da96a72c4680bea1f7dd511ddd61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:39.995607 11019 catalog_manager.cc:4841] TS 1ef48122fac9499caa7150f5a811f04e (127.14.164.3:42513): tablet f84e0751d9eb4b108dfefbd0db987183 (table test-table [id=01d7222c4783432f816d649fa462cdba]) successfully deleted
W20260810 07:23:39.997541 11022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:33291 (error 108)
I20260810 07:23:39.997850 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 11321
I20260810 07:23:40.002017 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.001851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:33291 (local address 127.0.0.1:57640)
0810 07:23:40.001927 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.001979 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 07:23:40.002403 11022 catalog_manager.cc:4584] TS 456703dbac0e447aa7fd42a3082c354c (127.14.164.1:33291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f84e0751d9eb4b108dfefbd0db987183: Network error: Client connection negotiation failed: client connection to 127.14.164.1:33291: connect: Connection refused (error 111)
I20260810 07:23:40.009864 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 11511
I20260810 07:23:40.009944 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.009789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:39313 (local address 127.0.0.1:59302)
0810 07:23:40.009874 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.009904 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.2:39313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:23:40.026424 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 10997
2026-08-10T07:23:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2509 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T07:23:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:40Z Disabled control of system clock
I20260810 07:23:40.062624 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:34177
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:34177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:23:40.146137 11756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:40.146391 11756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:40.146414 11756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:40.147727 11756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:40.147776 11756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:40.147789 11756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:40.147802 11756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:40.147814 11756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:40.149298 11756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:34177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:34177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:40.149573 11756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:40.149889 11756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:40.154810 11765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.154817 11769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.154930 11767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:40.155118 11756 server_base.cc:1034] running on GCE node
I20260810 07:23:40.155409 11756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:40.155717 11756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:40.156909 11756 hybrid_clock.cc:648] HybridClock initialized: now 1786346620156873 us; error 30 us; skew 500 ppm
I20260810 07:23:40.158211 11756 webserver.cc:460] Webserver started at http://127.14.164.62:39695/ using document root <none> and password file <none>
I20260810 07:23:40.158466 11756 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:40.158545 11756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:40.158692 11756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:40.159772 11756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "b2937ffa63e04a0f94e80d61006d8d34"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "2dfb5d6a60668e41a8457e099c70238d"
server_key_iv: "8279c4950ffbc3102cc2d2df3a45dc68"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.160161 11756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "b2937ffa63e04a0f94e80d61006d8d34"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "2dfb5d6a60668e41a8457e099c70238d"
server_key_iv: "8279c4950ffbc3102cc2d2df3a45dc68"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.161588 11756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:40.162336 11776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.162516 11756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:23:40.162626 11756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "b2937ffa63e04a0f94e80d61006d8d34"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "2dfb5d6a60668e41a8457e099c70238d"
server_key_iv: "8279c4950ffbc3102cc2d2df3a45dc68"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.162700 11756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:40.176483 11756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:40.176846 11756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:40.177011 11756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:40.181043 11756 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:34177
I20260810 07:23:40.181080 11849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:34177 every 8 connection(s)
I20260810 07:23:40.181437 11756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:40.182441 11850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:40.184695 11850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34: Bootstrap starting.
I20260810 07:23:40.185271 11850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:40.185518 11850 log.cc:824] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:40.186180 11850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34: No bootstrap required, opened a new log
I20260810 07:23:40.187472 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 11756
I20260810 07:23:40.187583 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:40.187779 14992 external_mini_cluster.cc:1428] Setting key 07d177404a4ca46b826f5423b65a09a7
I20260810 07:23:40.188172 11850 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2937ffa63e04a0f94e80d61006d8d34" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 34177 } }
I20260810 07:23:40.188354 11850 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:40.188396 11850 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2937ffa63e04a0f94e80d61006d8d34, State: Initialized, Role: FOLLOWER
I20260810 07:23:40.188529 11850 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2937ffa63e04a0f94e80d61006d8d34" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 34177 } }
I20260810 07:23:40.188634 11850 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:40.188676 11850 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:40.188741 11850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:40.189486 11850 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2937ffa63e04a0f94e80d61006d8d34" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 34177 } }
I20260810 07:23:40.189616 11850 leader_election.cc:302] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2937ffa63e04a0f94e80d61006d8d34; no voters:
I20260810 07:23:40.189812 11850 leader_election.cc:288] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:40.189934 11857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:40.190079 11850 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:40.190095 11857 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [term 1 LEADER]: Becoming Leader. State: Replica: b2937ffa63e04a0f94e80d61006d8d34, State: Running, Role: LEADER
I20260810 07:23:40.190267 11857 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2937ffa63e04a0f94e80d61006d8d34" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 34177 } }
I20260810 07:23:40.190642 11859 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2937ffa63e04a0f94e80d61006d8d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2937ffa63e04a0f94e80d61006d8d34" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 34177 } } }
I20260810 07:23:40.190631 11857 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2937ffa63e04a0f94e80d61006d8d34. Latest consensus state: current_term: 1 leader_uuid: "b2937ffa63e04a0f94e80d61006d8d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2937ffa63e04a0f94e80d61006d8d34" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 34177 } } }
I20260810 07:23:40.190892 11859 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:40.190912 11857 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:40.191267 11878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:40.191994 11878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:40.194294 11878 catalog_manager.cc:1347] Generated new cluster ID: 9907210e4b1049f197d8cc2e00735dc3
I20260810 07:23:40.194357 11878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:40.194840 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.188396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.14.164.62:34177)
0810 07:23:40.188688 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.188697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.188749 (+ 52us) server_negotiation.cc:407] Connection header received
0810 07:23:40.188866 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.188874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.188947 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.189093 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:40.189325 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.191250 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.192475 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.193122 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.193250 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.194035 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:40.194039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:40.194040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:40.194258 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:40.194303 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:40.194318 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:40.194401 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.194535 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.194623 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
I20260810 07:23:40.209136 11878 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:40.209872 11878 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:40.214222 11878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34: Generated new TSK 0
I20260810 07:23:40.214428 11878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:40.220500 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:34177
--builtin_ntp_servers=127.14.164.20:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:23:40.310128 11891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:40.310387 11891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:40.310426 11891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:40.311609 11891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:40.311677 11891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:40.311724 11891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:40.313220 11891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:40.313496 11891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:40.313828 11891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:40.319900 11898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.319968 11899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.320034 11903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:40.320325 11891 server_base.cc:1034] running on GCE node
I20260810 07:23:40.320580 11891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:40.320873 11891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:40.322046 11891 hybrid_clock.cc:648] HybridClock initialized: now 1786346620322009 us; error 56 us; skew 500 ppm
I20260810 07:23:40.323366 11891 webserver.cc:460] Webserver started at http://127.14.164.1:40505/ using document root <none> and password file <none>
I20260810 07:23:40.323637 11891 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:40.323707 11891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:40.323858 11891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:40.325114 11891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "1257e61fb49048fc937e3fe50db2550c"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "3246e6b3435615583db5863f0cce4059"
server_key_iv: "7499297bb71b765b67a726c6d9763c4a"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.325498 11891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "1257e61fb49048fc937e3fe50db2550c"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "3246e6b3435615583db5863f0cce4059"
server_key_iv: "7499297bb71b765b67a726c6d9763c4a"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.326938 11891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:23:40.327728 11911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.327943 11891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:40.328058 11891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "1257e61fb49048fc937e3fe50db2550c"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "3246e6b3435615583db5863f0cce4059"
server_key_iv: "7499297bb71b765b67a726c6d9763c4a"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.328136 11891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:40.347635 11891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:40.348014 11891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:40.348181 11891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:40.348457 11891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:40.348918 11891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:40.348976 11891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.349014 11891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:40.349062 11891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.355409 11891 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:39965
I20260810 07:23:40.355450 12076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:39965 every 8 connection(s)
I20260810 07:23:40.355840 11891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:40.357121 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 11891
I20260810 07:23:40.357234 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:40.357439 14992 external_mini_cluster.cc:1428] Setting key 186ccc99697c3f72179fac1526e46a73
I20260810 07:23:40.358520 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:34177
--builtin_ntp_servers=127.14.164.20:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:23:40.362192 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.356761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:53991 (local address 127.14.164.62:34177)
0810 07:23:40.356832 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.356835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.357064 (+ 229us) server_negotiation.cc:407] Connection header received
0810 07:23:40.357688 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.357689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.357720 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.357802 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:40.358736 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.359246 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.360503 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.360703 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.360839 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.361819 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:40.361820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:40.361821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:40.361826 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:40.361842 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:40.361847 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:40.361955 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.362070 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.362107 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:23:40.362426 12082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.356678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:34177 (local address 127.14.164.1:53991)
0810 07:23:40.356907 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.356951 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.357566 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.357904 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.357908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.357919 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:40.358635 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.358642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.359361 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.359362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.360400 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.360404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.360853 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.360854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.361010 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.361644 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:40.361645 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:40.361676 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:40.361702 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:40.361960 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:40.361967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:40.361971 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.362057 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.362169 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":131,"threads_started":1}
I20260810 07:23:40.363065 12077 heartbeater.cc:347] Connected to a master server at 127.14.164.62:34177
I20260810 07:23:40.363180 12077 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:40.363379 12077 heartbeater.cc:511] Master 127.14.164.62:34177 requested a full tablet report, sending...
I20260810 07:23:40.364023 11802 ts_manager.cc:194] Registered new tserver with Master: 1257e61fb49048fc937e3fe50db2550c (127.14.164.1:39965)
I20260810 07:23:40.365106 11802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.1:53991
W20260810 07:23:40.448000 12083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:40.448241 12083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:40.448266 12083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:40.449548 12083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:40.449595 12083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:40.449622 12083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:40.450975 12083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:40.451182 12083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:40.451455 12083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:40.456925 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.456895 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:40.457129 12083 server_base.cc:1034] running on GCE node
W20260810 07:23:40.456923 12096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:40.457499 12083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:40.457719 12083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:40.458961 12083 hybrid_clock.cc:648] HybridClock initialized: now 1786346620458937 us; error 44 us; skew 500 ppm
I20260810 07:23:40.460337 12083 webserver.cc:460] Webserver started at http://127.14.164.2:44283/ using document root <none> and password file <none>
I20260810 07:23:40.460613 12083 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:40.460697 12083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:40.460865 12083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:40.461993 12083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "12471af249984235aa13875d7a3c51ec"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "7b78a7cebc874b7e64828cebf043da10"
server_key_iv: "9438eb43aefbabaffe425a75341f4044"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.462369 12083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "12471af249984235aa13875d7a3c51ec"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "7b78a7cebc874b7e64828cebf043da10"
server_key_iv: "9438eb43aefbabaffe425a75341f4044"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.463747 12083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:40.464563 12105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.464774 12083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:40.464903 12083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "12471af249984235aa13875d7a3c51ec"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "7b78a7cebc874b7e64828cebf043da10"
server_key_iv: "9438eb43aefbabaffe425a75341f4044"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.464988 12083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:40.503826 12083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:40.504194 12083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:40.504360 12083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:40.504633 12083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:40.505031 12083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:40.505091 12083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.505158 12083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:40.505187 12083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.511168 12083 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:42807
I20260810 07:23:40.511219 12284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:42807 every 8 connection(s)
I20260810 07:23:40.511565 12083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:40.515730 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 12083
I20260810 07:23:40.515849 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:40.516084 14992 external_mini_cluster.cc:1428] Setting key 51528de496ad61544ea8a6c1da69f03a
I20260810 07:23:40.517081 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.512359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:57537 (local address 127.14.164.62:34177)
0810 07:23:40.512455 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.512457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.512690 (+ 233us) server_negotiation.cc:407] Connection header received
0810 07:23:40.513206 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.513207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.513216 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.513285 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:40.513939 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.514433 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.515507 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.515704 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.515764 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.516788 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:40.516789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:40.516789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:40.516795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:40.516814 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:40.516819 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:40.516882 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.516994 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.517019 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:23:40.517215 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:34177
--builtin_ntp_servers=127.14.164.20:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:23:40.517339 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.512350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:34177 (local address 127.14.164.2:57537)
0810 07:23:40.512558 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.512601 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.513111 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.513338 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.513341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.513349 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:40.513861 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.513866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.514523 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.514524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.515406 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.515409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.515796 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.515797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.515968 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.516586 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:40.516587 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:40.516615 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:40.516638 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:40.516898 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:40.516905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:40.516908 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.516988 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.517105 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260810 07:23:40.517987 12286 heartbeater.cc:347] Connected to a master server at 127.14.164.62:34177
I20260810 07:23:40.518110 12286 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:40.518308 12286 heartbeater.cc:511] Master 127.14.164.62:34177 requested a full tablet report, sending...
I20260810 07:23:40.518741 11802 ts_manager.cc:194] Registered new tserver with Master: 12471af249984235aa13875d7a3c51ec (127.14.164.2:42807)
I20260810 07:23:40.519129 11802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.2:57537
W20260810 07:23:40.605353 12290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:40.605616 12290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:40.605640 12290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:40.606868 12290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:40.606917 12290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:40.606945 12290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:40.608336 12290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:40.608619 12290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:40.608971 12290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:40.614571 12301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.614660 12305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:40.614585 12303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:40.614876 12290 server_base.cc:1034] running on GCE node
I20260810 07:23:40.615181 12290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:40.615393 12290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:40.616539 12290 hybrid_clock.cc:648] HybridClock initialized: now 1786346620616525 us; error 27 us; skew 500 ppm
I20260810 07:23:40.617949 12290 webserver.cc:460] Webserver started at http://127.14.164.3:41533/ using document root <none> and password file <none>
I20260810 07:23:40.618165 12290 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:40.618207 12290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:40.618316 12290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:40.619379 12290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "23450a2235c04d21bf9ecab77bc3f6ce"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "597db9237da61a503767927dfa31cf46"
server_key_iv: "518a6373bff57dafda32752965a1918f"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.619695 12290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "23450a2235c04d21bf9ecab77bc3f6ce"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "597db9237da61a503767927dfa31cf46"
server_key_iv: "518a6373bff57dafda32752965a1918f"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.621088 12290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:40.621836 12313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.622031 12290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:40.622107 12290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "23450a2235c04d21bf9ecab77bc3f6ce"
format_stamp: "Formatted at 2026-08-10 07:23:40 on dist-test-slave-x5fp"
server_key: "597db9237da61a503767927dfa31cf46"
server_key_iv: "518a6373bff57dafda32752965a1918f"
server_key_version: "encryptionkey@0"
I20260810 07:23:40.622162 12290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:40.634423 12290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:40.634720 12290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:40.634826 12290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:40.635016 12290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:40.635382 12290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:40.635424 12290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.635484 12290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:40.635516 12290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:40.641957 12290 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:45531
I20260810 07:23:40.641999 12499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:45531 every 8 connection(s)
I20260810 07:23:40.642402 12290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:40.643826 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 12290
I20260810 07:23:40.643941 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:40.644153 14992 external_mini_cluster.cc:1428] Setting key 73579309578c307a1d4db857d01be56c
I20260810 07:23:40.648634 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.643278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:34367 (local address 127.14.164.62:34177)
0810 07:23:40.643364 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.643366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.643625 (+ 259us) server_negotiation.cc:407] Connection header received
0810 07:23:40.644446 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.644448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.644462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.644525 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:40.645514 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.646113 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.647120 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.647314 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.647375 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.648367 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:40.648369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:40.648369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:40.648374 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:40.648391 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:40.648395 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:40.648442 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.648542 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.648571 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:23:40.648916 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.643243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:34177 (local address 127.14.164.3:34367)
0810 07:23:40.643488 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.643528 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.644326 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.644606 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.644609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.644618 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:40.645385 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.645391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.646214 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.646214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.647019 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.647021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.647397 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.647397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.647519 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.648213 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:40.648214 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:40.648246 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:40.648270 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:40.648460 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:40.648465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:40.648468 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.648524 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.648613 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":132,"threads_started":1}
I20260810 07:23:40.649435 12501 heartbeater.cc:347] Connected to a master server at 127.14.164.62:34177
I20260810 07:23:40.649554 12501 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:40.649756 12501 heartbeater.cc:511] Master 127.14.164.62:34177 requested a full tablet report, sending...
I20260810 07:23:40.650187 11802 ts_manager.cc:194] Registered new tserver with Master: 23450a2235c04d21bf9ecab77bc3f6ce (127.14.164.3:45531)
I20260810 07:23:40.650576 11802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.3:34367
I20260810 07:23:40.656023 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:40.660846 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.657647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.14.164.62:34177)
0810 07:23:40.657744 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.657747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.657818 (+ 71us) server_negotiation.cc:407] Connection header received
0810 07:23:40.657900 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.657901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.657916 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.657995 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:40.658181 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.658690 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.659478 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.659677 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.659757 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.660509 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:40.660510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:40.660511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:40.660516 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:40.660537 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:40.660541 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:40.660641 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.660710 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.660746 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:23:40.662670 11802 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:40.664345 11802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:40.671545 12515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.666976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:42807 (local address 127.0.0.1:44578)
0810 07:23:40.667303 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.667314 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.667371 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.668051 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.668052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.668054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:40.668088 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.668089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.669211 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.669213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.670482 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.670485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.671238 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.671239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.671317 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.671330 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.671406 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.671464 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":88,"threads_started":1}
I20260810 07:23:40.671558 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.667103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.14.164.2:42807)
0810 07:23:40.667841 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.667844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.667854 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:40.667905 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.667907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.667957 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.668167 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:40.668239 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.669128 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.670584 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.671145 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.671238 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.671340 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.671420 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.671463 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":93,"threads_started":1}
I20260810 07:23:40.672055 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.666928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:39965 (local address 127.0.0.1:44250)
0810 07:23:40.667124 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.667173 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.667244 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.667698 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.667701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.667703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:40.667801 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.667804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.669373 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.669374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.670465 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.670468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.671784 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.671785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.671857 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.671862 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.671918 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.671964 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260810 07:23:40.672255 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.667451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51752 (local address 127.14.164.3:45531)
0810 07:23:40.667804 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.667809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.667821 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:23:40.667862 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.667865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.667880 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.667956 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:40.668240 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.669495 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.670696 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.671463 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.671529 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.671611 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.672118 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.672166 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":128,"threads_started":1}
I20260810 07:23:40.672376 12178 tablet_service.cc:1467] Processing CreateTablet for tablet b947df188d004f72acd8608fb30e9567 (DEFAULT_TABLE table=test-table [id=e810d3dc36b74c61aefd1d24dcd9e528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:40.672335 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.667170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:45531 (local address 127.0.0.1:51752)
0810 07:23:40.667467 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.667478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.667533 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.668034 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.668035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.668038 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:40.668146 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.668148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.669583 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.669584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.670613 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.670615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.671648 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.671649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.672015 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.672019 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.672083 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.672129 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":18,"thread_start_us":122,"threads_started":1}
I20260810 07:23:40.672775 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.667141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.14.164.1:39965)
0810 07:23:40.667493 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.667497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.667505 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:23:40.667540 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.667542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.667576 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.667688 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:40.667903 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.669274 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.670564 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.671393 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.671473 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.671569 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.672566 (+ 997us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.672618 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":102,"threads_started":1}
I20260810 07:23:40.672885 12178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b947df188d004f72acd8608fb30e9567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:40.673547 11976 tablet_service.cc:1467] Processing CreateTablet for tablet b947df188d004f72acd8608fb30e9567 (DEFAULT_TABLE table=test-table [id=e810d3dc36b74c61aefd1d24dcd9e528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:40.673952 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b947df188d004f72acd8608fb30e9567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:40.673928 12389 tablet_service.cc:1467] Processing CreateTablet for tablet b947df188d004f72acd8608fb30e9567 (DEFAULT_TABLE table=test-table [id=e810d3dc36b74c61aefd1d24dcd9e528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:40.674299 12389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b947df188d004f72acd8608fb30e9567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:40.675993 12522 tablet_bootstrap.cc:492] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Bootstrap starting.
I20260810 07:23:40.676678 12522 tablet_bootstrap.cc:654] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:40.677172 12522 log.cc:824] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:40.678059 12522 tablet_bootstrap.cc:492] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: No bootstrap required, opened a new log
I20260810 07:23:40.678149 12522 ts_tablet_manager.cc:1397] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:23:40.678251 12526 tablet_bootstrap.cc:492] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Bootstrap starting.
I20260810 07:23:40.678833 12526 tablet_bootstrap.cc:654] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:40.679088 12526 log.cc:824] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:40.679816 12526 tablet_bootstrap.cc:492] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: No bootstrap required, opened a new log
I20260810 07:23:40.679874 12526 ts_tablet_manager.cc:1397] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:40.680249 12522 raft_consensus.cc:348] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.680420 12522 raft_consensus.cc:374] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:40.680466 12522 raft_consensus.cc:729] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12471af249984235aa13875d7a3c51ec, State: Initialized, Role: FOLLOWER
I20260810 07:23:40.680594 12522 consensus_queue.cc:260] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.680895 12524 tablet_bootstrap.cc:492] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: Bootstrap starting.
I20260810 07:23:40.680977 12286 heartbeater.cc:503] Master 127.14.164.62:34177 was elected leader, sending a full tablet report...
I20260810 07:23:40.680936 12522 ts_tablet_manager.cc:1428] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:23:40.681679 12524 tablet_bootstrap.cc:654] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:40.682039 12524 log.cc:824] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:40.682633 12526 raft_consensus.cc:348] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.682765 12524 tablet_bootstrap.cc:492] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: No bootstrap required, opened a new log
I20260810 07:23:40.682796 12526 raft_consensus.cc:374] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:40.682811 12524 ts_tablet_manager.cc:1397] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:40.682848 12526 raft_consensus.cc:729] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23450a2235c04d21bf9ecab77bc3f6ce, State: Initialized, Role: FOLLOWER
I20260810 07:23:40.682984 12526 consensus_queue.cc:260] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.683296 12526 ts_tablet_manager.cc:1428] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 07:23:40.683364 12501 heartbeater.cc:503] Master 127.14.164.62:34177 was elected leader, sending a full tablet report...
I20260810 07:23:40.685047 12524 raft_consensus.cc:348] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.685175 12524 raft_consensus.cc:374] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:40.685199 12524 raft_consensus.cc:729] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1257e61fb49048fc937e3fe50db2550c, State: Initialized, Role: FOLLOWER
I20260810 07:23:40.685300 12524 consensus_queue.cc:260] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.685518 12524 ts_tablet_manager.cc:1428] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:40.685632 12077 heartbeater.cc:503] Master 127.14.164.62:34177 was elected leader, sending a full tablet report...
I20260810 07:23:40.710361 12531 raft_consensus.cc:482] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:40.710510 12531 raft_consensus.cc:504] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.710843 12531 leader_election.cc:288] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1257e61fb49048fc937e3fe50db2550c (127.14.164.1:39965), 12471af249984235aa13875d7a3c51ec (127.14.164.2:42807)
I20260810 07:23:40.714030 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.711178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:51955 (local address 127.14.164.2:42807)
0810 07:23:40.711255 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.711257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.711480 (+ 223us) server_negotiation.cc:407] Connection header received
0810 07:23:40.711557 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.711558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.711570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.711632 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:40.711797 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.712261 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.713289 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.713652 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.713749 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.713758 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.713913 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.713952 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:23:40.714073 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.711218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:60661 (local address 127.14.164.1:39965)
0810 07:23:40.711282 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.711284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.711294 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:40.711321 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.711322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.711332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.711421 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:40.711608 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.712138 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.713283 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.713678 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.713764 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.713775 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.713947 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.713991 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:23:40.714089 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.711049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:39965 (local address 127.14.164.3:60661)
0810 07:23:40.711092 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.711103 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.711147 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.711429 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.711431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.711433 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:40.711518 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.711520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.712228 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.712229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.713173 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.713175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.713787 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.713788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.713856 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.713859 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.713927 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.713971 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":3}
I20260810 07:23:40.714103 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.711049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:42807 (local address 127.14.164.3:51955)
0810 07:23:40.711411 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.711420 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:40.711493 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:40.711648 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:40.711649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:40.711651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:40.711727 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.711729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.712345 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.712346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.713195 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:40.713197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.713750 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.713751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:40.713832 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.713835 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:40.713903 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:23:40.713943 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":167,"thread_start_us":128,"threads_started":1}
I20260810 07:23:40.714476 12006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b947df188d004f72acd8608fb30e9567" candidate_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1257e61fb49048fc937e3fe50db2550c" is_pre_election: true
I20260810 07:23:40.714663 12006 raft_consensus.cc:2454] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23450a2235c04d21bf9ecab77bc3f6ce in term 0.
I20260810 07:23:40.714658 12214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b947df188d004f72acd8608fb30e9567" candidate_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12471af249984235aa13875d7a3c51ec" is_pre_election: true
I20260810 07:23:40.714807 12214 raft_consensus.cc:2454] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23450a2235c04d21bf9ecab77bc3f6ce in term 0.
I20260810 07:23:40.714974 12315 leader_election.cc:302] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1257e61fb49048fc937e3fe50db2550c, 23450a2235c04d21bf9ecab77bc3f6ce; no voters:
I20260810 07:23:40.715112 12531 raft_consensus.cc:2781] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:40.715176 12531 raft_consensus.cc:482] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:40.715197 12531 raft_consensus.cc:3037] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:40.716082 12531 raft_consensus.cc:504] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.716403 12006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b947df188d004f72acd8608fb30e9567" candidate_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1257e61fb49048fc937e3fe50db2550c"
I20260810 07:23:40.716432 12214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b947df188d004f72acd8608fb30e9567" candidate_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12471af249984235aa13875d7a3c51ec"
I20260810 07:23:40.716506 12214 raft_consensus.cc:3037] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:40.716498 12006 raft_consensus.cc:3037] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:40.717557 12006 raft_consensus.cc:2454] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23450a2235c04d21bf9ecab77bc3f6ce in term 1.
I20260810 07:23:40.717559 12214 raft_consensus.cc:2454] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23450a2235c04d21bf9ecab77bc3f6ce in term 1.
I20260810 07:23:40.716325 12531 leader_election.cc:288] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [CANDIDATE]: Term 1 election: Requested vote from peers 1257e61fb49048fc937e3fe50db2550c (127.14.164.1:39965), 12471af249984235aa13875d7a3c51ec (127.14.164.2:42807)
I20260810 07:23:40.717784 12315 leader_election.cc:302] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1257e61fb49048fc937e3fe50db2550c, 23450a2235c04d21bf9ecab77bc3f6ce; no voters:
I20260810 07:23:40.717927 12531 raft_consensus.cc:2781] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:40.718158 12531 raft_consensus.cc:686] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 1 LEADER]: Becoming Leader. State: Replica: 23450a2235c04d21bf9ecab77bc3f6ce, State: Running, Role: LEADER
I20260810 07:23:40.718282 12531 consensus_queue.cc:237] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } }
I20260810 07:23:40.719228 11801 catalog_manager.cc:5495] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 23450a2235c04d21bf9ecab77bc3f6ce (127.14.164.3). New cstate: current_term: 1 leader_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23450a2235c04d21bf9ecab77bc3f6ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 45531 } health_report { overall_health: HEALTHY } } }
I20260810 07:23:40.733605 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.731027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.14.164.3:45531)
0810 07:23:40.731088 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:40.731093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:40.731109 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:23:40.731147 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:40.731148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:40.731168 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:40.731230 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:40.731352 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.731837 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.732656 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:40.732976 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:40.733054 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:40.733262 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:40.733362 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:40.733462 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:40.733502 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 07:23:40.733511 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:23:40.735704 12214 raft_consensus.cc:1261] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 1 FOLLOWER]: Refusing update from remote peer 23450a2235c04d21bf9ecab77bc3f6ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:40.735743 12006 raft_consensus.cc:1261] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 1 FOLLOWER]: Refusing update from remote peer 23450a2235c04d21bf9ecab77bc3f6ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:40.736112 12531 consensus_queue.cc:1035] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "12471af249984235aa13875d7a3c51ec" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 42807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:40.736266 12538 consensus_queue.cc:1035] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "1257e61fb49048fc937e3fe50db2550c" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:40.739574 12550 mvcc.cc:205] Tried to move back new op lower bound from 7316875758530551808 to 7316875758463070208. Current Snapshot: MvccSnapshot[applied={T|T < 7316875758530551808}]
I20260810 07:23:40.741986 11802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:40.742105 11802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:40.742918 11802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2937ffa63e04a0f94e80d61006d8d34: Sending DeleteTablet for 3 replicas of tablet b947df188d004f72acd8608fb30e9567
I20260810 07:23:40.743287 12389 tablet_service.cc:1514] Processing DeleteTablet for tablet b947df188d004f72acd8608fb30e9567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:40 UTC) from {username='slave'} at 127.0.0.1:51752
I20260810 07:23:40.743283 11976 tablet_service.cc:1514] Processing DeleteTablet for tablet b947df188d004f72acd8608fb30e9567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:40 UTC) from {username='slave'} at 127.0.0.1:44250
I20260810 07:23:40.743542 12559 tablet_replica.cc:330] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c: stopping tablet replica
I20260810 07:23:40.743626 12178 tablet_service.cc:1514] Processing DeleteTablet for tablet b947df188d004f72acd8608fb30e9567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:40 UTC) from {username='slave'} at 127.0.0.1:44578
I20260810 07:23:40.743667 12559 raft_consensus.cc:2229] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:40.743788 12559 raft_consensus.cc:2258] T b947df188d004f72acd8608fb30e9567 P 1257e61fb49048fc937e3fe50db2550c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:40.743954 12562 tablet_replica.cc:330] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: stopping tablet replica
I20260810 07:23:40.744055 12562 raft_consensus.cc:2229] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:23:40.744207 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 11891
I20260810 07:23:40.766902 12563 tablet_replica.cc:330] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: stopping tablet replica
I20260810 07:23:40.767187 12563 raft_consensus.cc:2229] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:40.744307 12562 raft_consensus.cc:2258] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:40.767570 12563 raft_consensus.cc:2258] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:40.767993 12562 ts_tablet_manager.cc:1905] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:40.768151 12563 ts_tablet_manager.cc:1905] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:40.769809 12562 ts_tablet_manager.cc:1918] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:40.769879 12563 ts_tablet_manager.cc:1918] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:40.769970 12563 log.cc:1196] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/b947df188d004f72acd8608fb30e9567
I20260810 07:23:40.770067 12562 log.cc:1196] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/b947df188d004f72acd8608fb30e9567
I20260810 07:23:40.770265 12563 ts_tablet_manager.cc:1939] T b947df188d004f72acd8608fb30e9567 P 12471af249984235aa13875d7a3c51ec: Deleting consensus metadata
I20260810 07:23:40.770512 12562 ts_tablet_manager.cc:1939] T b947df188d004f72acd8608fb30e9567 P 23450a2235c04d21bf9ecab77bc3f6ce: Deleting consensus metadata
I20260810 07:23:40.770763 11780 catalog_manager.cc:4841] TS 12471af249984235aa13875d7a3c51ec (127.14.164.2:42807): tablet b947df188d004f72acd8608fb30e9567 (table test-table [id=e810d3dc36b74c61aefd1d24dcd9e528]) successfully deleted
I20260810 07:23:40.771025 11780 catalog_manager.cc:4841] TS 23450a2235c04d21bf9ecab77bc3f6ce (127.14.164.3:45531): tablet b947df188d004f72acd8608fb30e9567 (table test-table [id=e810d3dc36b74c61aefd1d24dcd9e528]) successfully deleted
W20260810 07:23:40.774169 11779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:39965 (error 108)
I20260810 07:23:40.774863 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:40.774606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:39965 (local address 127.0.0.1:44264)
0810 07:23:40.774759 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:40.774817 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:39965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260810 07:23:40.774987 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 12083
W20260810 07:23:40.782241 11779 catalog_manager.cc:4584] TS 1257e61fb49048fc937e3fe50db2550c (127.14.164.1:39965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b947df188d004f72acd8608fb30e9567: Network error: Client connection negotiation failed: client connection to 127.14.164.1:39965: connect: Connection refused (error 111)
I20260810 07:23:40.786921 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 12290
I20260810 07:23:40.798236 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 11756
2026-08-10T07:23:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (776 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:40 dist-test-slave-x5fp krb5kdc[12567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:40 dist-test-slave-x5fp krb5kdc[12567](info): set up 2 sockets
Aug 10 07:23:40 dist-test-slave-x5fp krb5kdc[12567](info): commencing operation
krb5kdc: starting...
W20260810 07:23:42.089550 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:23:42.223388 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39239
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:39239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:42.326042 12588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:42.326277 12588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:42.326318 12588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:42.328118 12588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:42.328178 12588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:42.328203 12588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:42.328236 12588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:42.328264 12588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:42.330667 12588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:39239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:39239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:42.330947 12588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:42.331250 12588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:42.338050 12594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:42.338236 12593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.338253 12588 server_base.cc:1034] running on GCE node
W20260810 07:23:42.338063 12599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.338665 12588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:42.339005 12588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:42.340183 12588 hybrid_clock.cc:648] HybridClock initialized: now 1786346622340160 us; error 36 us; skew 500 ppm
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:42.343907 12588 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:23:42.344394 12588 webserver.cc:460] Webserver started at http://127.14.164.62:38435/ using document root <none> and password file <none>
I20260810 07:23:42.344653 12588 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:42.344722 12588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:42.344921 12588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:42.346035 12588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "5c2b9479796e4ffd875a4b15e6bcd31c"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "2001b19f5f7d727fd9151b57506cc58c"
server_key_iv: "93701182af106eff8ee566f63cdd9531"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.346422 12588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "5c2b9479796e4ffd875a4b15e6bcd31c"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "2001b19f5f7d727fd9151b57506cc58c"
server_key_iv: "93701182af106eff8ee566f63cdd9531"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.347819 12588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:23:42.348675 12611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.348984 12588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:42.349105 12588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "5c2b9479796e4ffd875a4b15e6bcd31c"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "2001b19f5f7d727fd9151b57506cc58c"
server_key_iv: "93701182af106eff8ee566f63cdd9531"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.349193 12588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:42.360924 12588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:42.363432 12588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:42.363603 12588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:42.367681 12588 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:39239
I20260810 07:23:42.367741 12679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:39239 every 8 connection(s)
I20260810 07:23:42.368140 12588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:42.369154 12680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:42.370275 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 12588
I20260810 07:23:42.370396 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:42.370630 14992 external_mini_cluster.cc:1428] Setting key 0a2b9bb575575855f33f317d7a46efa6
I20260810 07:23:42.372335 12680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c: Bootstrap starting.
I20260810 07:23:42.373106 12680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:42.373442 12680 log.cc:824] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:42.374258 12680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c: No bootstrap required, opened a new log
I20260810 07:23:42.376578 12680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39239 } }
I20260810 07:23:42.376770 12680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:42.376850 12680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2b9479796e4ffd875a4b15e6bcd31c, State: Initialized, Role: FOLLOWER
I20260810 07:23:42.377017 12680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39239 } }
I20260810 07:23:42.377087 12680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:42.377146 12680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:42.377194 12680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:42.377843 12680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39239 } }
I20260810 07:23:42.377980 12680 leader_election.cc:302] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c2b9479796e4ffd875a4b15e6bcd31c; no voters:
I20260810 07:23:42.378161 12680 leader_election.cc:288] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:42.378196 12691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 1 FOLLOWER]: Leader election won for term 1
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:42.378329 12691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [term 1 LEADER]: Becoming Leader. State: Replica: 5c2b9479796e4ffd875a4b15e6bcd31c, State: Running, Role: LEADER
I20260810 07:23:42.378450 12680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:42.378427 12691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39239 } }
I20260810 07:23:42.378757 12691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c2b9479796e4ffd875a4b15e6bcd31c. Latest consensus state: current_term: 1 leader_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39239 } } }
I20260810 07:23:42.378845 12691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:42.379336 12693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2b9479796e4ffd875a4b15e6bcd31c" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 39239 } } }
I20260810 07:23:42.379438 12693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:42.379755 12695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:42.380595 12695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:42.382151 12695 catalog_manager.cc:1347] Generated new cluster ID: b8c7870d84294f2f98b2d119a24d4765
I20260810 07:23:42.382207 12695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:42.382926 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.371191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.14.164.62:39239)
0810 07:23:42.371382 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.371392 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.371425 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:23:42.372042 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.372052 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.372118 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.372282 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:42.373023 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.374374 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.375144 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.375768 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.375907 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.379008 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:42.379012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:42.379012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:42.379024 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:42.381211 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.381636 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.381639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.381640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.381754 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.381989 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.381990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.381990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.382298 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:42.382462 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.382578 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.382724 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260810 07:23:42.395272 12695 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:42.395817 12695 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:42.398705 12695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c: Generated new TSK 0
I20260810 07:23:42.398934 12695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.164.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
Entry for principal kudu/127.14.164.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab.
I20260810 07:23:42.432144 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39239
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:42.518976 12723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:42.519214 12723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:42.519239 12723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:42.520442 12723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:42.520488 12723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:42.520517 12723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:42.522106 12723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.1.keytab
--principal=kudu/127.14.164.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:42.522408 12723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:42.522665 12723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:42.528139 12733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:42.528180 12736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.528426 12723 server_base.cc:1034] running on GCE node
W20260810 07:23:42.528571 12732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.528875 12723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:42.529204 12723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:42.530388 12723 hybrid_clock.cc:648] HybridClock initialized: now 1786346622530362 us; error 44 us; skew 500 ppm
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:42.533195 12723 init.cc:377] Logged in from keytab as kudu/127.14.164.1@KRBTEST.COM (short username kudu)
I20260810 07:23:42.533671 12723 webserver.cc:460] Webserver started at http://127.14.164.1:33347/ using document root <none> and password file <none>
I20260810 07:23:42.533939 12723 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:42.534011 12723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:42.534168 12723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:42.535266 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "dd7226d2222a46b1a245aa361e564629"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "a03f9457e419d80a4eff23c2ef456456"
server_key_iv: "ae907e12a0f7ba68e075dff8ec513952"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.535648 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "dd7226d2222a46b1a245aa361e564629"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "a03f9457e419d80a4eff23c2ef456456"
server_key_iv: "ae907e12a0f7ba68e075dff8ec513952"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.537075 12723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:23:42.537825 12744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.538067 12723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:42.538172 12723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "dd7226d2222a46b1a245aa361e564629"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "a03f9457e419d80a4eff23c2ef456456"
server_key_iv: "ae907e12a0f7ba68e075dff8ec513952"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.538259 12723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:42.556530 12723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:42.558964 12723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:42.559090 12723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:42.559295 12723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:42.559651 12723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:42.559690 12723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.559715 12723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:42.559728 12723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.565835 12723 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:41177
I20260810 07:23:42.565932 12920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:41177 every 8 connection(s)
I20260810 07:23:42.566267 12723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:42.568770 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 12723
I20260810 07:23:42.568888 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:42.569101 14992 external_mini_cluster.cc:1428] Setting key 8a15be7dce33f22064d509e8c56f4e7c
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.1@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:42.577454 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.567090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:60235 (local address 127.14.164.62:39239)
0810 07:23:42.567168 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.567171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.567400 (+ 229us) server_negotiation.cc:407] Connection header received
0810 07:23:42.567907 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.567908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.567917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.567973 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:42.569309 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.569953 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.571515 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.571837 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.571927 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.574346 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:42.574348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:42.574349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:42.574359 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:42.575928 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.576361 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.576363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.576363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.576431 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.576700 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.576701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.576702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.576883 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:42.576965 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.577259 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.577356 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:23:42.577658 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.567049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39239 (local address 127.14.164.1:60235)
0810 07:23:42.567254 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.567310 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.567811 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.568030 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.568034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.568850 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:42.569178 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.569186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.570049 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.570050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.571406 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.571410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.571944 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.571945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.572055 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.572661 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:42.572669 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:42.574237 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:42.576024 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:42.576033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:42.576035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:42.576278 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:42.576509 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:42.576510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:42.576510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:42.576623 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:42.576956 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:42.576964 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:42.577117 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.577227 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.577408 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260810 07:23:42.578219 12922 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39239
I20260810 07:23:42.578344 12922 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:42.578569 12922 heartbeater.cc:511] Master 127.14.164.62:39239 requested a full tablet report, sending...
I20260810 07:23:42.579159 12631 ts_manager.cc:194] Registered new tserver with Master: dd7226d2222a46b1a245aa361e564629 (127.14.164.1:41177)
WARNING: no policy specified for kudu/127.14.164.2@KRBTEST.COM; defaulting to no policy
I20260810 07:23:42.580014 12631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.1@KRBTEST.COM'} at 127.14.164.1:60235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
Entry for principal kudu/127.14.164.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab.
I20260810 07:23:42.600685 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39239
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:42.686817 12931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:42.687048 12931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:42.687073 12931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:42.688309 12931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:42.688360 12931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:42.688387 12931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:42.689929 12931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.2.keytab
--principal=kudu/127.14.164.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:42.690188 12931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:42.690424 12931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:42.695834 12938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:42.695915 12939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.696035 12931 server_base.cc:1034] running on GCE node
W20260810 07:23:42.695854 12944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.696312 12931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:42.696637 12931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:42.697819 12931 hybrid_clock.cc:648] HybridClock initialized: now 1786346622697800 us; error 29 us; skew 500 ppm
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:42.700436 12931 init.cc:377] Logged in from keytab as kudu/127.14.164.2@KRBTEST.COM (short username kudu)
I20260810 07:23:42.700992 12931 webserver.cc:460] Webserver started at http://127.14.164.2:41339/ using document root <none> and password file <none>
I20260810 07:23:42.701228 12931 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:42.701299 12931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:42.701458 12931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:42.702561 12931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "95c2a8ebf1f44c4395151cf9a83bbe53"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "dfd39bc8c15093593e5426f0073d098a"
server_key_iv: "57018bf11f4b39f1e4af437805c1f5ce"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.702952 12931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "95c2a8ebf1f44c4395151cf9a83bbe53"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "dfd39bc8c15093593e5426f0073d098a"
server_key_iv: "57018bf11f4b39f1e4af437805c1f5ce"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.704314 12931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:42.705224 12951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.705461 12931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:42.705577 12931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "95c2a8ebf1f44c4395151cf9a83bbe53"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "dfd39bc8c15093593e5426f0073d098a"
server_key_iv: "57018bf11f4b39f1e4af437805c1f5ce"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.705655 12931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:42.717100 12931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:42.719393 12931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:42.719566 12931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:42.719833 12931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:42.720171 12931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:42.720229 12931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.720274 12931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:42.720299 12931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.727102 12931 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:37521
I20260810 07:23:42.727176 13122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:37521 every 8 connection(s)
I20260810 07:23:42.727540 12931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.2@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:42.737311 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 12931
I20260810 07:23:42.737440 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:42.737589 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.728423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:33595 (local address 127.14.164.62:39239)
0810 07:23:42.728480 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.728482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.728636 (+ 154us) server_negotiation.cc:407] Connection header received
0810 07:23:42.729270 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.729271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.729282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.729347 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:42.730359 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.730851 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.731879 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.732095 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.732153 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.734630 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:42.734632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:42.734633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:42.734641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:42.735971 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.736424 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.736426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.736426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.736486 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.736759 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.736767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.736767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.736896 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:42.736976 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.737358 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.737496 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:23:42.737661 14992 external_mini_cluster.cc:1428] Setting key f5f9b1e2eb7ab973147e0cda2d1723a0
I20260810 07:23:42.737831 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.728331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39239 (local address 127.14.164.2:33595)
0810 07:23:42.728508 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.728555 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.729146 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.729431 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.729435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.730039 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:42.730246 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.730251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.730956 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.730957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.731789 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.731792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.732176 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.732177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.732284 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.732928 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:42.732937 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:42.734483 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:42.736063 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:42.736071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:42.736074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:42.736319 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:42.736563 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:42.736563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:42.736564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:42.736675 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:42.736967 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:42.736975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:42.737166 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.737308 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.737510 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260810 07:23:42.738370 13123 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39239
I20260810 07:23:42.738478 13123 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:42.738672 13123 heartbeater.cc:511] Master 127.14.164.62:39239 requested a full tablet report, sending...
I20260810 07:23:42.739179 12631 ts_manager.cc:194] Registered new tserver with Master: 95c2a8ebf1f44c4395151cf9a83bbe53 (127.14.164.2:37521)
I20260810 07:23:42.739594 12631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.2@KRBTEST.COM'} at 127.14.164.2:33595
WARNING: no policy specified for kudu/127.14.164.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
Entry for principal kudu/127.14.164.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab.
I20260810 07:23:42.768157 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39239
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:42.856429 13135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:42.856670 13135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:42.856698 13135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:42.858086 13135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:42.858141 13135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:42.858170 13135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:42.859757 13135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.3.keytab
--principal=kudu/127.14.164.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:39239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:42.860018 13135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:42.860280 13135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:42.865608 13142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:42.865618 13145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:42.865635 13143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:42.866227 13135 server_base.cc:1034] running on GCE node
I20260810 07:23:42.866485 13135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:42.866760 13135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:42.867980 13135 hybrid_clock.cc:648] HybridClock initialized: now 1786346622867955 us; error 40 us; skew 500 ppm
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:42.870746 13135 init.cc:377] Logged in from keytab as kudu/127.14.164.3@KRBTEST.COM (short username kudu)
I20260810 07:23:42.871210 13135 webserver.cc:460] Webserver started at http://127.14.164.3:38647/ using document root <none> and password file <none>
I20260810 07:23:42.871461 13135 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:42.871512 13135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:42.871647 13135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:42.872912 13135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "7366a1e085d0478485fd2e66dc4f87ce"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "f3ed77e8566df59a4e45d18e268fa8dc"
server_key_iv: "1d9ecbcf38c3f110ae548adfa981985d"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.873302 13135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "7366a1e085d0478485fd2e66dc4f87ce"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "f3ed77e8566df59a4e45d18e268fa8dc"
server_key_iv: "1d9ecbcf38c3f110ae548adfa981985d"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.874747 13135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:23:42.875662 13161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.875891 13135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:42.876012 13135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "7366a1e085d0478485fd2e66dc4f87ce"
format_stamp: "Formatted at 2026-08-10 07:23:42 on dist-test-slave-x5fp"
server_key: "f3ed77e8566df59a4e45d18e268fa8dc"
server_key_iv: "1d9ecbcf38c3f110ae548adfa981985d"
server_key_version: "encryptionkey@0"
I20260810 07:23:42.876096 13135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:42.885144 13135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:42.887552 13135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:42.887740 13135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:42.887996 13135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:42.888353 13135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:42.888413 13135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.888459 13135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:42.888487 13135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:42.894842 13135 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:35887
I20260810 07:23:42.894892 13340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:35887 every 8 connection(s)
I20260810 07:23:42.895241 13135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:23:42 dist-test-slave-x5fp krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346622, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.3@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:23:42.904847 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 13135
I20260810 07:23:42.904973 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:42.905225 14992 external_mini_cluster.cc:1428] Setting key d9c75dc27c47dfb0646ffba40ca582f6
I20260810 07:23:42.906606 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.896008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:54541 (local address 127.14.164.62:39239)
0810 07:23:42.896095 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.896098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.896404 (+ 306us) server_negotiation.cc:407] Connection header received
0810 07:23:42.896980 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.896982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.896992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.897048 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:42.898038 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.898573 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.899741 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.899935 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.900001 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.902633 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:42.902635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:42.902635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:42.902644 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:42.904486 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.905162 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.905164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.905164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.905234 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.905705 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.905706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.905707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.905883 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:42.905964 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.906392 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.906495 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:23:42.906921 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.895979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:39239 (local address 127.14.164.3:54541)
0810 07:23:42.896179 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.896280 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.896874 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.897106 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.897109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.897728 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:42.897937 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.897943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.898693 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.898694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.899633 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.899637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.900047 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.900048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.900166 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.900952 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:42.900961 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:42.902518 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:42.904594 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:42.904606 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:42.904610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:42.905048 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:42.905404 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:42.905405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:42.905405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:42.905592 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:42.905996 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:42.906007 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:42.906213 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.906328 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.906621 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260810 07:23:42.907449 13342 heartbeater.cc:347] Connected to a master server at 127.14.164.62:39239
I20260810 07:23:42.907577 13342 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:42.907801 13342 heartbeater.cc:511] Master 127.14.164.62:39239 requested a full tablet report, sending...
I20260810 07:23:42.908299 12631 ts_manager.cc:194] Registered new tserver with Master: 7366a1e085d0478485fd2e66dc4f87ce (127.14.164.3:35887)
I20260810 07:23:42.908782 12631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.164.3@KRBTEST.COM'} at 127.14.164.3:54541
I20260810 07:23:42.918004 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:42.926649 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.919588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.14.164.62:39239)
0810 07:23:42.919702 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.919705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.919810 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:23:42.919911 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.919912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.919927 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.919993 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:42.920621 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.921216 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.922027 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.922249 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.922340 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.923778 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:42.923780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:42.923780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:42.923788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:42.925364 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.925747 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.925748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.925748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.925810 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:42.926018 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:42.926019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:42.926019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:42.926183 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:42.926295 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.926444 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.926528 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:23:42.928283 12631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:42.929939 12631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:42.937556 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.933133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.14.164.2:37521)
0810 07:23:42.933356 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.933362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.933372 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:42.933424 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.933427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.933445 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.933845 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:42.933946 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.934702 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.936220 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.937084 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.937202 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.937269 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.937412 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.937465 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
I20260810 07:23:42.937551 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.932825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:37521 (local address 127.0.0.1:38356)
0810 07:23:42.933127 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.933137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.933179 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.933665 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.933670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.933673 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:42.933756 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.933759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.934816 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.934817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.936113 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.936115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.937205 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.937206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.937293 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.937307 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.937383 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.937452 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":124,"threads_started":1}
I20260810 07:23:42.937779 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.932924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.14.164.1:41177)
0810 07:23:42.933405 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.933409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.933419 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:23:42.933466 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.933468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.933486 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.933592 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:42.933874 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.935095 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.936349 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.937098 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.937168 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.937238 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.937592 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.937640 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":99,"threads_started":1}
I20260810 07:23:42.937832 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.932804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:41177 (local address 127.0.0.1:57154)
0810 07:23:42.932976 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.933021 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.933090 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.933680 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.933681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.933684 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:42.933774 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.933776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.935225 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.935225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.936253 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.936255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.937276 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.937277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.937389 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.937394 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.937449 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.937492 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":54,"threads_started":1}
I20260810 07:23:42.938697 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.932829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:35887 (local address 127.0.0.1:50096)
0810 07:23:42.933219 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.933227 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.933399 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.933602 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.933816 (+ 214us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.933819 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:42.933867 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.933869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.936323 (+ 2454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.936323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.937067 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.937069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.938468 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.938469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.938535 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.938539 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.938588 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.938630 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":118,"threads_started":1}
I20260810 07:23:42.938838 12814 tablet_service.cc:1467] Processing CreateTablet for tablet 3e04141650544eb4908d8ac64f11a251 (DEFAULT_TABLE table=test-table [id=513e4dcd7cc340dbb515b83450ef1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:42.938825 13020 tablet_service.cc:1467] Processing CreateTablet for tablet 3e04141650544eb4908d8ac64f11a251 (DEFAULT_TABLE table=test-table [id=513e4dcd7cc340dbb515b83450ef1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:42.939040 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.933116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.14.164.3:35887)
0810 07:23:42.933345 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.933351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.933367 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:23:42.933476 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.933479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.933495 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.933587 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:42.933968 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.935092 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.937153 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.938296 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.938371 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.938432 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.938905 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.938955 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":98,"threads_started":1}
I20260810 07:23:42.939268 13020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e04141650544eb4908d8ac64f11a251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:42.939270 12814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e04141650544eb4908d8ac64f11a251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:42.939671 13233 tablet_service.cc:1467] Processing CreateTablet for tablet 3e04141650544eb4908d8ac64f11a251 (DEFAULT_TABLE table=test-table [id=513e4dcd7cc340dbb515b83450ef1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:42.940023 13233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e04141650544eb4908d8ac64f11a251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:42.942395 13374 tablet_bootstrap.cc:492] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Bootstrap starting.
I20260810 07:23:42.943110 13374 tablet_bootstrap.cc:654] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:42.943333 13376 tablet_bootstrap.cc:492] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: Bootstrap starting.
I20260810 07:23:42.943529 13377 tablet_bootstrap.cc:492] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Bootstrap starting.
I20260810 07:23:42.944010 13374 log.cc:824] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:42.944058 13376 tablet_bootstrap.cc:654] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:42.944197 13377 tablet_bootstrap.cc:654] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:42.944576 13376 log.cc:824] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:42.944873 13377 log.cc:824] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:42.945024 13374 tablet_bootstrap.cc:492] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: No bootstrap required, opened a new log
I20260810 07:23:42.945101 13374 ts_tablet_manager.cc:1397] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:23:42.945551 13376 tablet_bootstrap.cc:492] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: No bootstrap required, opened a new log
I20260810 07:23:42.945631 13376 ts_tablet_manager.cc:1397] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:42.945650 13377 tablet_bootstrap.cc:492] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: No bootstrap required, opened a new log
I20260810 07:23:42.945724 13377 ts_tablet_manager.cc:1397] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:42.947211 13374 raft_consensus.cc:348] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.947293 13377 raft_consensus.cc:348] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.947412 13374 raft_consensus.cc:374] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:42.947450 13374 raft_consensus.cc:729] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c2a8ebf1f44c4395151cf9a83bbe53, State: Initialized, Role: FOLLOWER
I20260810 07:23:42.947453 13377 raft_consensus.cc:374] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:42.947494 13377 raft_consensus.cc:729] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7366a1e085d0478485fd2e66dc4f87ce, State: Initialized, Role: FOLLOWER
I20260810 07:23:42.947618 13374 consensus_queue.cc:260] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.947631 13376 raft_consensus.cc:348] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.947794 13376 raft_consensus.cc:374] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:42.947853 13376 raft_consensus.cc:729] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7226d2222a46b1a245aa361e564629, State: Initialized, Role: FOLLOWER
I20260810 07:23:42.947949 13374 ts_tablet_manager.cc:1428] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:42.947956 13376 consensus_queue.cc:260] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.948069 13123 heartbeater.cc:503] Master 127.14.164.62:39239 was elected leader, sending a full tablet report...
I20260810 07:23:42.948228 13376 ts_tablet_manager.cc:1428] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:42.948382 12922 heartbeater.cc:503] Master 127.14.164.62:39239 was elected leader, sending a full tablet report...
I20260810 07:23:42.948167 13377 consensus_queue.cc:260] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.949055 13377 ts_tablet_manager.cc:1428] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:42.949250 13342 heartbeater.cc:503] Master 127.14.164.62:39239 was elected leader, sending a full tablet report...
I20260810 07:23:42.963990 13386 raft_consensus.cc:482] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:42.964119 13386 raft_consensus.cc:504] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.964514 13386 leader_election.cc:288] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95c2a8ebf1f44c4395151cf9a83bbe53 (127.14.164.2:37521), 7366a1e085d0478485fd2e66dc4f87ce (127.14.164.3:35887)
I20260810 07:23:42.968670 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.964881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:49239 (local address 127.14.164.3:35887)
0810 07:23:42.964997 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.964999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.965115 (+ 116us) server_negotiation.cc:407] Connection header received
0810 07:23:42.965143 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.965144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.965151 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.965214 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:42.965358 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.966072 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.967613 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.968212 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.968287 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.968297 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.968548 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.968592 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 07:23:42.968670 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.964820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:36031 (local address 127.14.164.2:37521)
0810 07:23:42.964923 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.964925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.964934 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:42.964965 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.964966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.964977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.965034 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:42.965246 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.966067 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.967612 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.968205 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.968288 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.968297 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.968547 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.968592 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 07:23:42.968742 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.964690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:37521 (local address 127.14.164.1:36031)
0810 07:23:42.964778 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.964790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.964835 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.965068 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.965069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.965071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:42.965157 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.965159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.966170 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.966171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.967501 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.967505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.968327 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.968328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.968437 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.968445 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.968524 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.968594 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":20}
I20260810 07:23:42.969065 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.964793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:35887 (local address 127.14.164.1:49239)
0810 07:23:42.965003 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:42.965031 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:42.965097 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:42.965218 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:42.965219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:42.965220 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:42.965291 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.965292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.966170 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.966171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.967501 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:42.967505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.968327 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.968328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:42.968437 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.968445 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:23:42.968524 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:23:42.968594 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":155,"thread_start_us":144,"threads_started":1}
I20260810 07:23:42.969185 13058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e04141650544eb4908d8ac64f11a251" candidate_uuid: "dd7226d2222a46b1a245aa361e564629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" is_pre_election: true
I20260810 07:23:42.969410 13058 raft_consensus.cc:2454] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7226d2222a46b1a245aa361e564629 in term 0.
I20260810 07:23:42.969471 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e04141650544eb4908d8ac64f11a251" candidate_uuid: "dd7226d2222a46b1a245aa361e564629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7366a1e085d0478485fd2e66dc4f87ce" is_pre_election: true
I20260810 07:23:42.969635 13264 raft_consensus.cc:2454] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7226d2222a46b1a245aa361e564629 in term 0.
I20260810 07:23:42.969674 12747 leader_election.cc:302] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95c2a8ebf1f44c4395151cf9a83bbe53, dd7226d2222a46b1a245aa361e564629; no voters:
I20260810 07:23:42.969918 13386 raft_consensus.cc:2781] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:42.969977 13386 raft_consensus.cc:482] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:42.969995 13386 raft_consensus.cc:3037] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:42.970773 13386 raft_consensus.cc:504] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.970928 13386 leader_election.cc:288] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [CANDIDATE]: Term 1 election: Requested vote from peers 95c2a8ebf1f44c4395151cf9a83bbe53 (127.14.164.2:37521), 7366a1e085d0478485fd2e66dc4f87ce (127.14.164.3:35887)
I20260810 07:23:42.971132 13058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e04141650544eb4908d8ac64f11a251" candidate_uuid: "dd7226d2222a46b1a245aa361e564629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53"
I20260810 07:23:42.971145 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e04141650544eb4908d8ac64f11a251" candidate_uuid: "dd7226d2222a46b1a245aa361e564629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7366a1e085d0478485fd2e66dc4f87ce"
I20260810 07:23:42.971240 13058 raft_consensus.cc:3037] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:42.971262 13264 raft_consensus.cc:3037] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:42.972110 13058 raft_consensus.cc:2454] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7226d2222a46b1a245aa361e564629 in term 1.
I20260810 07:23:42.972227 13264 raft_consensus.cc:2454] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7226d2222a46b1a245aa361e564629 in term 1.
I20260810 07:23:42.972290 12747 leader_election.cc:302] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95c2a8ebf1f44c4395151cf9a83bbe53, dd7226d2222a46b1a245aa361e564629; no voters:
I20260810 07:23:42.972425 13386 raft_consensus.cc:2781] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:42.972616 13386 raft_consensus.cc:686] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [term 1 LEADER]: Becoming Leader. State: Replica: dd7226d2222a46b1a245aa361e564629, State: Running, Role: LEADER
I20260810 07:23:42.972759 13386 consensus_queue.cc:237] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } }
I20260810 07:23:42.973752 12630 catalog_manager.cc:5495] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7226d2222a46b1a245aa361e564629 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "dd7226d2222a46b1a245aa361e564629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7226d2222a46b1a245aa361e564629" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 41177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 } health_report { overall_health: UNKNOWN } } }
W20260810 07:23:42.978251 13124 tablet.cc:2369] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:42.985854 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:42.983428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.14.164.1:41177)
0810 07:23:42.983484 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:42.983486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:42.983517 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:23:42.983615 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:42.983616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:42.983634 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:42.983700 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:42.983826 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.984297 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.985142 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:42.985342 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:42.985411 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:42.985542 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:42.985647 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:42.985711 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:42.985754 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:23:42.985776 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:23:42.987816 13058 raft_consensus.cc:1261] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 1 FOLLOWER]: Refusing update from remote peer dd7226d2222a46b1a245aa361e564629: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:42.987818 13264 raft_consensus.cc:1261] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 1 FOLLOWER]: Refusing update from remote peer dd7226d2222a46b1a245aa361e564629: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:42.988236 13393 consensus_queue.cc:1035] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7366a1e085d0478485fd2e66dc4f87ce" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:42.988402 13386 consensus_queue.cc:1035] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95c2a8ebf1f44c4395151cf9a83bbe53" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 37521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:42.990954 13399 mvcc.cc:205] Tried to move back new op lower bound from 7316875767755091968 to 7316875767697203200. Current Snapshot: MvccSnapshot[applied={T|T < 7316875767755091968}]
I20260810 07:23:42.991539 13401 mvcc.cc:205] Tried to move back new op lower bound from 7316875767755091968 to 7316875767697203200. Current Snapshot: MvccSnapshot[applied={T|T < 7316875767755091968}]
I20260810 07:23:42.991814 13402 mvcc.cc:205] Tried to move back new op lower bound from 7316875767755091968 to 7316875767697203200. Current Snapshot: MvccSnapshot[applied={T|T < 7316875767755091968}]
I20260810 07:23:42.993743 12631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:42.993840 12631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:42.994680 12631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c2b9479796e4ffd875a4b15e6bcd31c: Sending DeleteTablet for 3 replicas of tablet 3e04141650544eb4908d8ac64f11a251
I20260810 07:23:42.995059 13233 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e04141650544eb4908d8ac64f11a251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:42 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:50096
I20260810 07:23:42.995174 13020 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e04141650544eb4908d8ac64f11a251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:42 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:38356
I20260810 07:23:42.995464 13412 tablet_replica.cc:330] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: stopping tablet replica
I20260810 07:23:42.995493 12814 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e04141650544eb4908d8ac64f11a251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:42 UTC) from {username='kudu', principal='kudu/127.14.164.62@KRBTEST.COM'} at 127.0.0.1:57154
I20260810 07:23:42.995597 13412 raft_consensus.cc:2229] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:42.995765 13412 raft_consensus.cc:2258] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:42.995792 13414 tablet_replica.cc:330] T 3e04141650544eb4908d8ac64f11a251 P dd7226d2222a46b1a245aa361e564629: stopping tablet replica
I20260810 07:23:42.995887 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 12723
I20260810 07:23:43.000778 13412 ts_tablet_manager.cc:1905] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:43.002908 13412 ts_tablet_manager.cc:1918] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:43.002990 13412 log.cc:1196] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/3e04141650544eb4908d8ac64f11a251
I20260810 07:23:43.003237 13412 ts_tablet_manager.cc:1939] T 3e04141650544eb4908d8ac64f11a251 P 7366a1e085d0478485fd2e66dc4f87ce: Deleting consensus metadata
I20260810 07:23:43.004179 12614 catalog_manager.cc:4841] TS 7366a1e085d0478485fd2e66dc4f87ce (127.14.164.3:35887): tablet 3e04141650544eb4908d8ac64f11a251 (table test-table [id=513e4dcd7cc340dbb515b83450ef1754]) successfully deleted
I20260810 07:23:43.012326 13413 tablet_replica.cc:330] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: stopping tablet replica
I20260810 07:23:43.012542 13413 raft_consensus.cc:2229] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:43.012758 13413 raft_consensus.cc:2258] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:43.013988 13413 ts_tablet_manager.cc:1905] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:43.016276 13413 ts_tablet_manager.cc:1918] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:43.016390 13413 log.cc:1196] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/3e04141650544eb4908d8ac64f11a251
I20260810 07:23:43.016780 13413 ts_tablet_manager.cc:1939] T 3e04141650544eb4908d8ac64f11a251 P 95c2a8ebf1f44c4395151cf9a83bbe53: Deleting consensus metadata
I20260810 07:23:43.017351 12612 catalog_manager.cc:4841] TS 95c2a8ebf1f44c4395151cf9a83bbe53 (127.14.164.2:37521): tablet 3e04141650544eb4908d8ac64f11a251 (table test-table [id=513e4dcd7cc340dbb515b83450ef1754]) successfully deleted
W20260810 07:23:43.019423 12615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:41177 (error 108)
I20260810 07:23:43.020044 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.019906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:41177 (local address 127.0.0.1:57182)
0810 07:23:43.019966 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.020012 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:23:43.020143 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 12931
W20260810 07:23:43.020223 12615 catalog_manager.cc:4584] TS dd7226d2222a46b1a245aa361e564629 (127.14.164.1:41177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e04141650544eb4908d8ac64f11a251: Network error: Client connection negotiation failed: client connection to 127.14.164.1:41177: connect: Connection refused (error 111)
I20260810 07:23:43.039065 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 13135
I20260810 07:23:43.052296 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 12588
2026-08-10T07:23:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2251 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T07:23:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:43Z Disabled control of system clock
I20260810 07:23:43.091187 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35567
--webserver_interface=127.14.164.62
--webserver_port=0
--builtin_ntp_servers=127.14.164.20:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:35567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:23:43.177352 13421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:43.177603 13421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:43.177626 13421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:43.178817 13421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:43.178864 13421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:43.178880 13421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:43.178893 13421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:43.178906 13421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:43.180459 13421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:35567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:35567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:43.180787 13421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:43.181131 13421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:43.186475 13427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:43.186523 13428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:43.186748 13421 server_base.cc:1034] running on GCE node
W20260810 07:23:43.186475 13430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:43.187153 13421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:43.187490 13421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:43.188673 13421 hybrid_clock.cc:648] HybridClock initialized: now 1786346623188654 us; error 50 us; skew 500 ppm
I20260810 07:23:43.190035 13421 webserver.cc:460] Webserver started at http://127.14.164.62:41215/ using document root <none> and password file <none>
I20260810 07:23:43.190317 13421 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:43.190402 13421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:43.190563 13421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:43.191687 13421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "99d0fbe12b1045f7a02597cef75108e0"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "ee00b5bc08a7e0f966de860e39fa7f8a"
server_key_iv: "4ad26f0576ba394d4364f4160bc10d5b"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.192078 13421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "99d0fbe12b1045f7a02597cef75108e0"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "ee00b5bc08a7e0f966de860e39fa7f8a"
server_key_iv: "4ad26f0576ba394d4364f4160bc10d5b"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.193544 13421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 07:23:43.194343 13437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.194537 13421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:43.194650 13421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "99d0fbe12b1045f7a02597cef75108e0"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "ee00b5bc08a7e0f966de860e39fa7f8a"
server_key_iv: "4ad26f0576ba394d4364f4160bc10d5b"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.194730 13421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:43.208585 13421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:43.208984 13421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:43.209159 13421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:43.213335 13421 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:35567
I20260810 07:23:43.213389 13521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:35567 every 8 connection(s)
I20260810 07:23:43.213814 13421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:43.214887 13522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:43.216452 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 13421
I20260810 07:23:43.216595 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:43.216841 14992 external_mini_cluster.cc:1428] Setting key c42a9f96228dcad34cf4ac2413d055a0
I20260810 07:23:43.217834 13522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0: Bootstrap starting.
I20260810 07:23:43.218544 13522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:43.218873 13522 log.cc:824] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:43.219627 13522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0: No bootstrap required, opened a new log
I20260810 07:23:43.222136 13522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0fbe12b1045f7a02597cef75108e0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35567 } }
I20260810 07:23:43.222328 13522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:43.222358 13522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d0fbe12b1045f7a02597cef75108e0, State: Initialized, Role: FOLLOWER
I20260810 07:23:43.222482 13522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0fbe12b1045f7a02597cef75108e0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35567 } }
I20260810 07:23:43.222579 13522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:43.222625 13522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:43.222672 13522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:43.223130 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.217461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.14.164.62:35567)
0810 07:23:43.217656 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.217663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.217688 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:23:43.218147 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.218154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.218216 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.218343 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:43.218662 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.219804 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.220946 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.221366 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.221461 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.222264 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:43.222268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:43.222268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:43.222609 (+ 341us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:43.222656 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:43.222671 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:43.222762 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.222871 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.222954 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260810 07:23:43.223379 13522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0fbe12b1045f7a02597cef75108e0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35567 } }
I20260810 07:23:43.223491 13522 leader_election.cc:302] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99d0fbe12b1045f7a02597cef75108e0; no voters:
I20260810 07:23:43.223637 13522 leader_election.cc:288] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:43.223717 13531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:43.223892 13531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [term 1 LEADER]: Becoming Leader. State: Replica: 99d0fbe12b1045f7a02597cef75108e0, State: Running, Role: LEADER
I20260810 07:23:43.223899 13522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:43.224030 13531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0fbe12b1045f7a02597cef75108e0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35567 } }
I20260810 07:23:43.224408 13534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99d0fbe12b1045f7a02597cef75108e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0fbe12b1045f7a02597cef75108e0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35567 } } }
I20260810 07:23:43.224411 13535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99d0fbe12b1045f7a02597cef75108e0. Latest consensus state: current_term: 1 leader_uuid: "99d0fbe12b1045f7a02597cef75108e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0fbe12b1045f7a02597cef75108e0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 35567 } } }
I20260810 07:23:43.224588 13534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:43.224602 13535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:43.225003 13549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:43.225523 13549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:43.227389 13549 catalog_manager.cc:1347] Generated new cluster ID: 46f602ef54f84d398fa2b721b8353269
I20260810 07:23:43.227449 13549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:43.268191 13549 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:43.269091 13549 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:43.281662 13549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0: Generated new TSK 0
I20260810 07:23:43.281991 13549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:43.292997 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.1:0
--local_ip_for_outbound_sockets=127.14.164.1
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35567
--builtin_ntp_servers=127.14.164.20:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:23:43.379323 13562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:43.379551 13562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:43.379576 13562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:43.381072 13562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:43.381129 13562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:43.381157 13562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.1
I20260810 07:23:43.382609 13562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.164.1
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.1
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:43.382879 13562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:43.383195 13562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:43.388813 13574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:43.388986 13562 server_base.cc:1034] running on GCE node
W20260810 07:23:43.388813 13572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:43.389098 13571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:43.389353 13562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:43.389654 13562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:43.390832 13562 hybrid_clock.cc:648] HybridClock initialized: now 1786346623390811 us; error 31 us; skew 500 ppm
I20260810 07:23:43.392336 13562 webserver.cc:460] Webserver started at http://127.14.164.1:36831/ using document root <none> and password file <none>
I20260810 07:23:43.392645 13562 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:43.392699 13562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:43.392918 13562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:43.394198 13562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data/instance:
uuid: "c4a9a1d63c694156a761736cbc0c8a06"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "11311ef3ad82e5ffd7bd63336396ade2"
server_key_iv: "a441bf1e130545d5171f2911b4340b81"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.394738 13562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance:
uuid: "c4a9a1d63c694156a761736cbc0c8a06"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "11311ef3ad82e5ffd7bd63336396ade2"
server_key_iv: "a441bf1e130545d5171f2911b4340b81"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.396334 13562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:23:43.397286 13583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.397533 13562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:43.397656 13562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal
uuid: "c4a9a1d63c694156a761736cbc0c8a06"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "11311ef3ad82e5ffd7bd63336396ade2"
server_key_iv: "a441bf1e130545d5171f2911b4340b81"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.397740 13562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:43.417270 13562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:43.417708 13562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:43.417892 13562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:43.418160 13562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:43.418605 13562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:43.418663 13562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.418711 13562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:43.418730 13562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.425197 13562 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.1:46019
I20260810 07:23:43.425244 13745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.1:46019 every 8 connection(s)
I20260810 07:23:43.425601 13562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:43.429876 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 13562
I20260810 07:23:43.429987 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:43.430202 14992 external_mini_cluster.cc:1428] Setting key 3b1b34d987a8cfd5fd97491949bc87c8
I20260810 07:23:43.431388 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.426351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:33439 (local address 127.14.164.62:35567)
0810 07:23:43.426435 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.426438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.426647 (+ 209us) server_negotiation.cc:407] Connection header received
0810 07:23:43.427165 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.427166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.427180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.427248 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:43.427942 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.428437 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.429491 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.429737 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.429814 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.431055 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:43.431056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:43.431057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:43.431062 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:43.431078 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:43.431082 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:43.431159 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.431276 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.431307 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:23:43.431454 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.2:0
--local_ip_for_outbound_sockets=127.14.164.2
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35567
--builtin_ntp_servers=127.14.164.20:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:23:43.431661 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.426315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35567 (local address 127.14.164.1:33439)
0810 07:23:43.426513 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.426555 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:43.427079 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:43.427303 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:43.427306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:43.427315 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:43.427830 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.427835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.428523 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.428524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.429412 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.429415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.429824 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.429824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.429987 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.430849 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:43.430850 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:43.430890 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:43.430921 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:43.431174 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:43.431180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:43.431184 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:23:43.431272 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 07:23:43.431391 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260810 07:23:43.432343 13746 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35567
I20260810 07:23:43.432475 13746 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:43.432705 13746 heartbeater.cc:511] Master 127.14.164.62:35567 requested a full tablet report, sending...
I20260810 07:23:43.433406 13461 ts_manager.cc:194] Registered new tserver with Master: c4a9a1d63c694156a761736cbc0c8a06 (127.14.164.1:46019)
I20260810 07:23:43.434105 13461 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.1:33439
W20260810 07:23:43.518947 13750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:43.519280 13750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:43.519323 13750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:43.520525 13750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:43.520572 13750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:43.520597 13750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.2
I20260810 07:23:43.522075 13750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.164.2
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.2
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:43.522313 13750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:43.522578 13750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:43.528079 13756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:43.528142 13758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:43.528138 13762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:43.528362 13750 server_base.cc:1034] running on GCE node
I20260810 07:23:43.528574 13750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:43.528867 13750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:43.530025 13750 hybrid_clock.cc:648] HybridClock initialized: now 1786346623530011 us; error 32 us; skew 500 ppm
I20260810 07:23:43.531445 13750 webserver.cc:460] Webserver started at http://127.14.164.2:44963/ using document root <none> and password file <none>
I20260810 07:23:43.531716 13750 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:43.531831 13750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:43.531983 13750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:43.533185 13750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data/instance:
uuid: "413125d649bd4c9da1d9ae3f6ce0c42a"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "eb1498000b71efd2de75533849040a7a"
server_key_iv: "8a4c09fd00ee7f57e2148d8b3f64d395"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.533548 13750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance:
uuid: "413125d649bd4c9da1d9ae3f6ce0c42a"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "eb1498000b71efd2de75533849040a7a"
server_key_iv: "8a4c09fd00ee7f57e2148d8b3f64d395"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.535066 13750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:23:43.535915 13771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.536103 13750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:43.536214 13750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal
uuid: "413125d649bd4c9da1d9ae3f6ce0c42a"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "eb1498000b71efd2de75533849040a7a"
server_key_iv: "8a4c09fd00ee7f57e2148d8b3f64d395"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.536295 13750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:43.547719 13750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:43.548164 13750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:43.548357 13750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:43.548633 13750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:43.549067 13750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:43.549126 13750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.549162 13750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:43.549206 13750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.555456 13750 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.2:41629
I20260810 07:23:43.555503 13937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.2:41629 every 8 connection(s)
I20260810 07:23:43.555851 13750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:43.558615 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 13750
I20260810 07:23:43.558740 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:43.558996 14992 external_mini_cluster.cc:1428] Setting key c13eb22a215bc5f8f45f7912632e2050
I20260810 07:23:43.560158 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.164.3:0
--local_ip_for_outbound_sockets=127.14.164.3
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35567
--builtin_ntp_servers=127.14.164.20:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:23:43.561595 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.556652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.2:57793 (local address 127.14.164.62:35567)
0810 07:23:43.556704 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.556705 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.557015 (+ 310us) server_negotiation.cc:407] Connection header received
0810 07:23:43.557486 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.557488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.557500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.557566 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:43.558313 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.558920 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.560072 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.560327 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.560395 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.561344 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:43.561345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:43.561345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:43.561350 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:43.561365 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:43.561369 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:43.561417 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.561510 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.561534 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:23:43.561901 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.556615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35567 (local address 127.14.164.2:57793)
0810 07:23:43.556825 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.556868 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:43.557390 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:43.557649 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:43.557653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:43.557662 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:43.558217 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.558221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.559031 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.559032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.559986 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.559989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.560410 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.560411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.560575 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.561183 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:43.561184 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:43.561224 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:43.561255 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:43.561430 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:43.561434 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:43.561437 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:43.561505 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:23:43.561622 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260810 07:23:43.562469 13938 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35567
I20260810 07:23:43.562587 13938 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:43.562809 13938 heartbeater.cc:511] Master 127.14.164.62:35567 requested a full tablet report, sending...
I20260810 07:23:43.563231 13461 ts_manager.cc:194] Registered new tserver with Master: 413125d649bd4c9da1d9ae3f6ce0c42a (127.14.164.2:41629)
I20260810 07:23:43.563712 13461 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.2:57793
W20260810 07:23:43.647532 13943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:43.647747 13943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:43.647770 13943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:43.648914 13943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:43.648952 13943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:43.648975 13943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.164.3
I20260810 07:23:43.650282 13943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.164.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.164.3
--webserver_port=0
--tserver_master_addrs=127.14.164.62:35567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.164.3
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:43.650483 13943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:43.650754 13943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:43.656204 13957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:43.656195 13954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:43.656190 13959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:43.656234 13943 server_base.cc:1034] running on GCE node
I20260810 07:23:43.656596 13943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:43.656889 13943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:43.658078 13943 hybrid_clock.cc:648] HybridClock initialized: now 1786346623658068 us; error 42 us; skew 500 ppm
I20260810 07:23:43.659405 13943 webserver.cc:460] Webserver started at http://127.14.164.3:38669/ using document root <none> and password file <none>
I20260810 07:23:43.659675 13943 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:43.659747 13943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:43.659930 13943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:43.661131 13943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data/instance:
uuid: "27d374b014b140009e9bd0bd0aa29b2e"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "7a1231d0090a337ddef58bce168d05f2"
server_key_iv: "6566ad8109aeafbcf03cb020d952976d"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.661510 13943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance:
uuid: "27d374b014b140009e9bd0bd0aa29b2e"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "7a1231d0090a337ddef58bce168d05f2"
server_key_iv: "6566ad8109aeafbcf03cb020d952976d"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.662971 13943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:23:43.663759 13972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.663986 13943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:43.664101 13943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal
uuid: "27d374b014b140009e9bd0bd0aa29b2e"
format_stamp: "Formatted at 2026-08-10 07:23:43 on dist-test-slave-x5fp"
server_key: "7a1231d0090a337ddef58bce168d05f2"
server_key_iv: "6566ad8109aeafbcf03cb020d952976d"
server_key_version: "encryptionkey@0"
I20260810 07:23:43.664179 13943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:43.675045 13943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:43.675376 13943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:43.675532 13943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:43.675767 13943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:43.676154 13943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:43.676223 13943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.676270 13943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:43.676295 13943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:43.682883 13943 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.3:39529
I20260810 07:23:43.682930 14154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.3:39529 every 8 connection(s)
I20260810 07:23:43.683298 13943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:43.687446 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 13943
I20260810 07:23:43.687563 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:43.687748 14992 external_mini_cluster.cc:1428] Setting key 50381bfa23201957f4dfa1e43ca72fd8
I20260810 07:23:43.689594 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.684126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.3:55891 (local address 127.14.164.62:35567)
0810 07:23:43.684201 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.684205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.684454 (+ 249us) server_negotiation.cc:407] Connection header received
0810 07:23:43.685067 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.685068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.685078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.685130 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:43.685792 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.686295 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.688014 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.688276 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.688352 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.689389 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:43.689390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:43.689390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:43.689395 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:43.689412 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:43.689416 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:43.689460 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.689511 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.689534 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:23:43.689853 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.684097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:35567 (local address 127.14.164.3:55891)
0810 07:23:43.684328 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.684366 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:43.684973 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:43.685193 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:43.685196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:43.685204 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:43.685722 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.685726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.686426 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.686427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.687917 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.687920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.688381 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.688382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.688584 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.689257 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:23:43.689258 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:43.689288 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:23:43.689311 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:43.689460 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:43.689462 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:43.689465 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:43.689507 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 07:23:43.689587 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260810 07:23:43.690423 14156 heartbeater.cc:347] Connected to a master server at 127.14.164.62:35567
I20260810 07:23:43.690531 14156 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:43.690737 14156 heartbeater.cc:511] Master 127.14.164.62:35567 requested a full tablet report, sending...
I20260810 07:23:43.691156 13461 ts_manager.cc:194] Registered new tserver with Master: 27d374b014b140009e9bd0bd0aa29b2e (127.14.164.3:39529)
I20260810 07:23:43.691550 13461 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.164.3:55891
I20260810 07:23:43.699826 14992 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:43.704301 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.701420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.14.164.62:35567)
0810 07:23:43.701483 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.701489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.701608 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:23:43.701656 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.701656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.701665 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.701719 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:43.701854 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.702351 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.703155 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.703344 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.703409 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.704098 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:43.704099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:43.704100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:43.704104 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:43.704118 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:43.704122 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:43.704171 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.704219 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.704240 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:23:43.706090 13461 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:43.707657 13461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:43.714972 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.710381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.14.164.2:41629)
0810 07:23:43.710626 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.710632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.710999 (+ 367us) server_negotiation.cc:407] Connection header received
0810 07:23:43.711049 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.711051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.711068 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.711136 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:43.711349 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.712644 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.713890 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.714580 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.714656 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.714757 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.714830 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.714878 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":125,"threads_started":1}
I20260810 07:23:43.714970 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.710520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:39529 (local address 127.0.0.1:46228)
0810 07:23:43.710880 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.710888 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:43.710928 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:43.711463 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:43.711464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:43.711466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:43.711501 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.711502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.712377 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.712378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.713651 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.713654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.714648 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.714649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.714731 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.714745 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:23:43.714795 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:23:43.714858 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":71,"threads_started":1}
I20260810 07:23:43.714978 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.710277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:41629 (local address 127.0.0.1:48436)
0810 07:23:43.710737 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.710746 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:43.710794 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:43.711147 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:43.711147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:43.711149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:43.711237 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.711240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.712764 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.712765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.713786 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.713788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.714655 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.714656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.714735 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.714747 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 07:23:43.714818 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:23:43.714861 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":35,"thread_start_us":84,"threads_started":1}
I20260810 07:23:43.715696 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.710275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:46019 (local address 127.0.0.1:53550)
0810 07:23:43.710651 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:43.710697 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:43.710779 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:43.711118 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:43.711122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:43.711126 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:43.711229 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.711231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.713866 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.713866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.714502 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:43.714504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.715466 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.715467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:43.715528 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.715531 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:23:43.715592 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:23:43.715630 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260810 07:23:43.715725 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.710397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.14.164.1:46019)
0810 07:23:43.710667 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.710671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.710763 (+ 92us) server_negotiation.cc:407] Connection header received
0810 07:23:43.710878 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.710881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.710900 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.710998 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:43.711330 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.712651 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.714589 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.715362 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.715452 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.715539 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.715607 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.715646 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":144,"threads_started":1}
I20260810 07:23:43.715862 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:43.710776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.14.164.3:39529)
0810 07:23:43.711293 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:43.711299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:43.711308 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:43.711348 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:43.711349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:43.711363 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:43.711466 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:43.711558 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.712286 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.713739 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:43.714532 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:43.715475 (+ 943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:43.715586 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:43.715610 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 07:23:43.715656 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":81,"threads_started":1}
I20260810 07:23:43.716140 13844 tablet_service.cc:1467] Processing CreateTablet for tablet c36de6fa823f4b88a8409b4e14fdf2e8 (DEFAULT_TABLE table=test-table [id=f07a16caa9f246cabc6b6eea0946cf0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:43.716552 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36de6fa823f4b88a8409b4e14fdf2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:43.716437 14050 tablet_service.cc:1467] Processing CreateTablet for tablet c36de6fa823f4b88a8409b4e14fdf2e8 (DEFAULT_TABLE table=test-table [id=f07a16caa9f246cabc6b6eea0946cf0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:43.716435 13654 tablet_service.cc:1467] Processing CreateTablet for tablet c36de6fa823f4b88a8409b4e14fdf2e8 (DEFAULT_TABLE table=test-table [id=f07a16caa9f246cabc6b6eea0946cf0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:43.716828 14050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36de6fa823f4b88a8409b4e14fdf2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:43.716828 13654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36de6fa823f4b88a8409b4e14fdf2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:43.720351 14175 tablet_bootstrap.cc:492] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Bootstrap starting.
I20260810 07:23:43.720594 14176 tablet_bootstrap.cc:492] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Bootstrap starting.
I20260810 07:23:43.720786 14177 tablet_bootstrap.cc:492] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Bootstrap starting.
I20260810 07:23:43.721196 14175 tablet_bootstrap.cc:654] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:43.721346 14176 tablet_bootstrap.cc:654] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:43.721426 14177 tablet_bootstrap.cc:654] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:43.721606 14175 log.cc:824] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:43.721716 14176 log.cc:824] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:43.721760 14177 log.cc:824] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:43.722425 14175 tablet_bootstrap.cc:492] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: No bootstrap required, opened a new log
I20260810 07:23:43.722508 14175 ts_tablet_manager.cc:1397] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:43.722509 14177 tablet_bootstrap.cc:492] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: No bootstrap required, opened a new log
I20260810 07:23:43.722584 14177 ts_tablet_manager.cc:1397] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:43.722704 14176 tablet_bootstrap.cc:492] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: No bootstrap required, opened a new log
I20260810 07:23:43.722774 14176 ts_tablet_manager.cc:1397] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:43.725274 14177 raft_consensus.cc:348] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:43.725452 14177 raft_consensus.cc:374] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:43.725493 14177 raft_consensus.cc:729] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a9a1d63c694156a761736cbc0c8a06, State: Initialized, Role: FOLLOWER
I20260810 07:23:43.725427 14175 raft_consensus.cc:348] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:43.725427 14176 raft_consensus.cc:348] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:43.725592 14175 raft_consensus.cc:374] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:43.725592 14176 raft_consensus.cc:374] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:43.725630 14175 raft_consensus.cc:729] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413125d649bd4c9da1d9ae3f6ce0c42a, State: Initialized, Role: FOLLOWER
I20260810 07:23:43.725630 14176 raft_consensus.cc:729] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27d374b014b140009e9bd0bd0aa29b2e, State: Initialized, Role: FOLLOWER
I20260810 07:23:43.725636 14177 consensus_queue.cc:260] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:43.725863 14175 consensus_queue.cc:260] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:43.725919 14177 ts_tablet_manager.cc:1428] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:43.725960 14176 consensus_queue.cc:260] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:43.726047 13746 heartbeater.cc:503] Master 127.14.164.62:35567 was elected leader, sending a full tablet report...
I20260810 07:23:43.726158 14175 ts_tablet_manager.cc:1428] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:23:43.726197 14176 ts_tablet_manager.cc:1428] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:43.726274 13938 heartbeater.cc:503] Master 127.14.164.62:35567 was elected leader, sending a full tablet report...
I20260810 07:23:43.726298 14156 heartbeater.cc:503] Master 127.14.164.62:35567 was elected leader, sending a full tablet report...
W20260810 07:23:43.806434 13940 tablet.cc:2369] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:23:43.926486 13747 tablet.cc:2369] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:23:43.933993 14157 tablet.cc:2369] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:44.003439 14182 raft_consensus.cc:482] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:44.003592 14182 raft_consensus.cc:504] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:44.004045 14182 leader_election.cc:288] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413125d649bd4c9da1d9ae3f6ce0c42a (127.14.164.2:41629), 27d374b014b140009e9bd0bd0aa29b2e (127.14.164.3:39529)
I20260810 07:23:44.007583 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:44.004498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:56507 (local address 127.14.164.2:41629)
0810 07:23:44.004568 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:44.004570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:44.004653 (+ 83us) server_negotiation.cc:407] Connection header received
0810 07:23:44.004739 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:44.004741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:44.004754 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:44.004837 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:44.005046 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.005591 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.006608 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.007116 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.007180 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:44.007189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:44.007438 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 07:23:44.007485 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:23:44.007568 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:44.004398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.2:41629 (local address 127.14.164.1:56507)
0810 07:23:44.004564 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:44.004576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:44.004648 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:44.004862 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:44.004863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:44.004866 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:44.004953 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:44.004955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.005676 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.005677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:44.006498 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:44.006501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.007250 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.007252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:44.007333 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:44.007338 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:23:44.007416 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:23:44.007468 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260810 07:23:44.007830 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:44.004794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.164.1:59937 (local address 127.14.164.3:39529)
0810 07:23:44.004861 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:44.004863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:44.004872 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:44.004961 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:44.004963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:44.004973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:44.005030 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:44.005200 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.005684 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.006848 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.007334 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.007426 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:44.007435 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:44.007631 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:23:44.007742 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:23:44.008005 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:44.004631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.3:39529 (local address 127.14.164.1:59937)
0810 07:23:44.004772 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:44.004783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:44.004835 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:44.005048 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:44.005048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:44.005050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:44.005127 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:44.005128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.005792 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.005793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:44.006714 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:44.006717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.007446 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.007447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:44.007528 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:44.007533 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:23:44.007600 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:23:44.007806 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":54,"threads_started":1}
I20260810 07:23:44.008076 13868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36de6fa823f4b88a8409b4e14fdf2e8" candidate_uuid: "c4a9a1d63c694156a761736cbc0c8a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" is_pre_election: true
I20260810 07:23:44.008332 13868 raft_consensus.cc:2454] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a9a1d63c694156a761736cbc0c8a06 in term 0.
I20260810 07:23:44.008467 14087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36de6fa823f4b88a8409b4e14fdf2e8" candidate_uuid: "c4a9a1d63c694156a761736cbc0c8a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27d374b014b140009e9bd0bd0aa29b2e" is_pre_election: true
I20260810 07:23:44.008621 13587 leader_election.cc:302] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413125d649bd4c9da1d9ae3f6ce0c42a, c4a9a1d63c694156a761736cbc0c8a06; no voters:
I20260810 07:23:44.008656 14087 raft_consensus.cc:2454] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a9a1d63c694156a761736cbc0c8a06 in term 0.
I20260810 07:23:44.008824 14182 raft_consensus.cc:2781] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:44.008900 14182 raft_consensus.cc:482] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:44.008932 14182 raft_consensus.cc:3037] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:44.009785 14182 raft_consensus.cc:504] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:44.009912 14182 leader_election.cc:288] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [CANDIDATE]: Term 1 election: Requested vote from peers 413125d649bd4c9da1d9ae3f6ce0c42a (127.14.164.2:41629), 27d374b014b140009e9bd0bd0aa29b2e (127.14.164.3:39529)
I20260810 07:23:44.010201 14087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36de6fa823f4b88a8409b4e14fdf2e8" candidate_uuid: "c4a9a1d63c694156a761736cbc0c8a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27d374b014b140009e9bd0bd0aa29b2e"
I20260810 07:23:44.010201 13868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36de6fa823f4b88a8409b4e14fdf2e8" candidate_uuid: "c4a9a1d63c694156a761736cbc0c8a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a"
I20260810 07:23:44.010308 14087 raft_consensus.cc:3037] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:44.010308 13868 raft_consensus.cc:3037] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:44.011292 14087 raft_consensus.cc:2454] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a9a1d63c694156a761736cbc0c8a06 in term 1.
I20260810 07:23:44.011292 13868 raft_consensus.cc:2454] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a9a1d63c694156a761736cbc0c8a06 in term 1.
I20260810 07:23:44.011513 13587 leader_election.cc:302] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27d374b014b140009e9bd0bd0aa29b2e, c4a9a1d63c694156a761736cbc0c8a06; no voters:
I20260810 07:23:44.011636 14182 raft_consensus.cc:2781] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:44.011895 14182 raft_consensus.cc:686] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 1 LEADER]: Becoming Leader. State: Replica: c4a9a1d63c694156a761736cbc0c8a06, State: Running, Role: LEADER
I20260810 07:23:44.012020 14182 consensus_queue.cc:237] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } }
I20260810 07:23:44.012934 13461 catalog_manager.cc:5495] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a9a1d63c694156a761736cbc0c8a06 (127.14.164.1). New cstate: current_term: 1 leader_uuid: "c4a9a1d63c694156a761736cbc0c8a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a9a1d63c694156a761736cbc0c8a06" member_type: VOTER last_known_addr { host: "127.14.164.1" port: 46019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 } health_report { overall_health: UNKNOWN } } }
I20260810 07:23:44.076213 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:44.073651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.14.164.1:46019)
0810 07:23:44.073756 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:44.073758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:44.073767 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:23:44.073807 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:44.073808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:44.073829 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:44.073892 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:44.074089 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.074562 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.075308 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:44.075511 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:44.075582 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:44.075806 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:44.075913 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:44.075981 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:44.076061 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:23:44.076083 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:23:44.078400 14087 raft_consensus.cc:1261] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 1 FOLLOWER]: Refusing update from remote peer c4a9a1d63c694156a761736cbc0c8a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:44.078403 13868 raft_consensus.cc:1261] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 1 FOLLOWER]: Refusing update from remote peer c4a9a1d63c694156a761736cbc0c8a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:44.078783 14189 consensus_queue.cc:1035] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27d374b014b140009e9bd0bd0aa29b2e" member_type: VOTER last_known_addr { host: "127.14.164.3" port: 39529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:44.078931 14182 consensus_queue.cc:1035] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413125d649bd4c9da1d9ae3f6ce0c42a" member_type: VOTER last_known_addr { host: "127.14.164.2" port: 41629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:44.081617 14192 mvcc.cc:205] Tried to move back new op lower bound from 7316875772221968384 to 7316875771953848320. Current Snapshot: MvccSnapshot[applied={T|T < 7316875772221968384}]
I20260810 07:23:44.082012 14198 mvcc.cc:205] Tried to move back new op lower bound from 7316875772221968384 to 7316875771953848320. Current Snapshot: MvccSnapshot[applied={T|T < 7316875772221968384}]
I20260810 07:23:44.082252 14197 mvcc.cc:205] Tried to move back new op lower bound from 7316875772221968384 to 7316875771953848320. Current Snapshot: MvccSnapshot[applied={T|T < 7316875772221968384 or (T in {7316875772221968384})}]
I20260810 07:23:44.084347 13461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:44.084467 13461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:44.085304 13461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99d0fbe12b1045f7a02597cef75108e0: Sending DeleteTablet for 3 replicas of tablet c36de6fa823f4b88a8409b4e14fdf2e8
I20260810 07:23:44.085626 13654 tablet_service.cc:1514] Processing DeleteTablet for tablet c36de6fa823f4b88a8409b4e14fdf2e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:44 UTC) from {username='slave'} at 127.0.0.1:53550
I20260810 07:23:44.085747 13844 tablet_service.cc:1514] Processing DeleteTablet for tablet c36de6fa823f4b88a8409b4e14fdf2e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:44 UTC) from {username='slave'} at 127.0.0.1:48436
I20260810 07:23:44.085755 14050 tablet_service.cc:1514] Processing DeleteTablet for tablet c36de6fa823f4b88a8409b4e14fdf2e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:44 UTC) from {username='slave'} at 127.0.0.1:46228
I20260810 07:23:44.085855 14207 tablet_replica.cc:330] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: stopping tablet replica
I20260810 07:23:44.085961 14207 raft_consensus.cc:2229] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:23:44.086064 14207 raft_consensus.cc:2258] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:44.086347 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 13562
I20260810 07:23:44.086529 14207 ts_tablet_manager.cc:1905] T c36de6fa823f4b88a8409b4e14fdf2e8 P c4a9a1d63c694156a761736cbc0c8a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:44.090011 14209 tablet_replica.cc:330] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: stopping tablet replica
I20260810 07:23:44.090161 14209 raft_consensus.cc:2229] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:44.090310 14209 raft_consensus.cc:2258] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:44.090546 14211 tablet_replica.cc:330] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: stopping tablet replica
I20260810 07:23:44.090651 14211 raft_consensus.cc:2229] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:44.090741 14211 raft_consensus.cc:2258] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:44.091111 14209 ts_tablet_manager.cc:1905] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:44.092906 14209 ts_tablet_manager.cc:1918] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:44.093011 14209 log.cc:1196] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-1/wal/wals/c36de6fa823f4b88a8409b4e14fdf2e8
I20260810 07:23:44.093356 14209 ts_tablet_manager.cc:1939] T c36de6fa823f4b88a8409b4e14fdf2e8 P 413125d649bd4c9da1d9ae3f6ce0c42a: Deleting consensus metadata
I20260810 07:23:44.094213 13439 catalog_manager.cc:4841] TS 413125d649bd4c9da1d9ae3f6ce0c42a (127.14.164.2:41629): tablet c36de6fa823f4b88a8409b4e14fdf2e8 (table test-table [id=f07a16caa9f246cabc6b6eea0946cf0d]) successfully deleted
I20260810 07:23:44.094610 14211 ts_tablet_manager.cc:1905] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:44.096596 14211 ts_tablet_manager.cc:1918] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:44.096688 14211 log.cc:1196] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Deleting WAL directory at /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346363752364-14992-0/minicluster-data/ts-2/wal/wals/c36de6fa823f4b88a8409b4e14fdf2e8
I20260810 07:23:44.097012 14211 ts_tablet_manager.cc:1939] T c36de6fa823f4b88a8409b4e14fdf2e8 P 27d374b014b140009e9bd0bd0aa29b2e: Deleting consensus metadata
I20260810 07:23:44.097540 13439 catalog_manager.cc:4841] TS 27d374b014b140009e9bd0bd0aa29b2e (127.14.164.3:39529): tablet c36de6fa823f4b88a8409b4e14fdf2e8 (table test-table [id=f07a16caa9f246cabc6b6eea0946cf0d]) successfully deleted
W20260810 07:23:44.099756 13441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.164.1:46019 (error 108)
I20260810 07:23:44.100347 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 13750
I20260810 07:23:44.100421 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:44.100263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.1:46019 (local address 127.0.0.1:53558)
0810 07:23:44.100341 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:44.100391 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.1:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 07:23:44.105254 13441 catalog_manager.cc:4584] TS c4a9a1d63c694156a761736cbc0c8a06 (127.14.164.1:46019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c36de6fa823f4b88a8409b4e14fdf2e8: Network error: Client connection negotiation failed: client connection to 127.14.164.1:46019: connect: Connection refused (error 111)
I20260810 07:23:44.110129 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 13943
I20260810 07:23:44.124292 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 13421
2026-08-10T07:23:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1072 ms)
[----------] 6 tests from EncryptionPolicyTest (9844 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:44 dist-test-slave-x5fp krb5kdc[14219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:44 dist-test-slave-x5fp krb5kdc[14219](info): set up 2 sockets
Aug 10 07:23:44 dist-test-slave-x5fp krb5kdc[14219](info): commencing operation
krb5kdc: starting...
W20260810 07:23:45.391804 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:45 dist-test-slave-x5fp krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:45.529137 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46819
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:45.614503 14239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:45.614723 14239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:45.614748 14239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:45.615877 14239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:45.615919 14239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:45.615933 14239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:45.615947 14239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:45.615958 14239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:45.617712 14239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46819
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:45.617974 14239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:45.618232 14239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:45.623440 14253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:45.623500 14239 server_base.cc:1034] running on GCE node
W20260810 07:23:45.623503 14250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:45.623458 14249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:45.624047 14239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:45.624374 14239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:45.625545 14239 hybrid_clock.cc:648] HybridClock initialized: now 1786346625625531 us; error 40 us; skew 500 ppm
Aug 10 07:23:45 dist-test-slave-x5fp krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:45.628135 14239 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:45.628582 14239 webserver.cc:460] Webserver started at http://127.0.0.1:41701/ using document root <none> and password file <none>
I20260810 07:23:45.628867 14239 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:45.628952 14239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:45.629115 14239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:45.630280 14239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "4e58563943f4476bbf26fafa5cd3d495"
format_stamp: "Formatted at 2026-08-10 07:23:45 on dist-test-slave-x5fp"
server_key: "810b5a3df613ab5daa85e2537c77aa71"
server_key_iv: "4140a778c9f8535b9dc36b7f36c474b2"
server_key_version: "encryptionkey@0"
I20260810 07:23:45.630671 14239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "4e58563943f4476bbf26fafa5cd3d495"
format_stamp: "Formatted at 2026-08-10 07:23:45 on dist-test-slave-x5fp"
server_key: "810b5a3df613ab5daa85e2537c77aa71"
server_key_iv: "4140a778c9f8535b9dc36b7f36c474b2"
server_key_version: "encryptionkey@0"
I20260810 07:23:45.632027 14239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:45.632804 14265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:45.632995 14239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:45.633100 14239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "4e58563943f4476bbf26fafa5cd3d495"
format_stamp: "Formatted at 2026-08-10 07:23:45 on dist-test-slave-x5fp"
server_key: "810b5a3df613ab5daa85e2537c77aa71"
server_key_iv: "4140a778c9f8535b9dc36b7f36c474b2"
server_key_version: "encryptionkey@0"
I20260810 07:23:45.633184 14239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:45.661185 14239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:45.663306 14239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:45.663497 14239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:45.667754 14239 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46819
I20260810 07:23:45.667793 14342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46819 every 8 connection(s)
I20260810 07:23:45.668192 14239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:45.669189 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:45.671650 14344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495: Bootstrap starting.
I20260810 07:23:45.672279 14344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:45.672626 14344 log.cc:824] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:45.673446 14344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495: No bootstrap required, opened a new log
I20260810 07:23:45.675149 14344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e58563943f4476bbf26fafa5cd3d495" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46819 } }
I20260810 07:23:45.675315 14344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:45.675341 14344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e58563943f4476bbf26fafa5cd3d495, State: Initialized, Role: FOLLOWER
I20260810 07:23:45.675521 14344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e58563943f4476bbf26fafa5cd3d495" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46819 } }
I20260810 07:23:45.675606 14344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:45.675630 14344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:45.675660 14344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:45.676026 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 14239
I20260810 07:23:45.676175 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:45.676386 14992 external_mini_cluster.cc:1428] Setting key ab217017dc39817780afc879565d805b
I20260810 07:23:45.676412 14344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e58563943f4476bbf26fafa5cd3d495" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46819 } }
I20260810 07:23:45.676549 14344 leader_election.cc:302] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e58563943f4476bbf26fafa5cd3d495; no voters:
I20260810 07:23:45.676797 14344 leader_election.cc:288] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:45.676822 14352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:45.677063 14352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [term 1 LEADER]: Becoming Leader. State: Replica: 4e58563943f4476bbf26fafa5cd3d495, State: Running, Role: LEADER
I20260810 07:23:45.677170 14344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:45.677211 14352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e58563943f4476bbf26fafa5cd3d495" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46819 } }
I20260810 07:23:45.677856 14358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e58563943f4476bbf26fafa5cd3d495. Latest consensus state: current_term: 1 leader_uuid: "4e58563943f4476bbf26fafa5cd3d495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e58563943f4476bbf26fafa5cd3d495" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46819 } } }
I20260810 07:23:45.677971 14358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:45.678339 14355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e58563943f4476bbf26fafa5cd3d495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e58563943f4476bbf26fafa5cd3d495" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46819 } } }
I20260810 07:23:45.678469 14355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495 [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:45.679227 14380 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:45.679292 14380 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:45.679461 14369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:45.679617 14369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:45.681219 14369 catalog_manager.cc:1347] Generated new cluster ID: 76ea57b2c9614599a60e9c773cf9454a
I20260810 07:23:45.681327 14369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:23:45 dist-test-slave-x5fp krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:23:45.689821 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:45.677072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41217 (local address 127.0.0.1:46819)
0810 07:23:45.677313 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:45.677324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:45.677469 (+ 145us) server_negotiation.cc:407] Connection header received
0810 07:23:45.677624 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:45.677633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:45.677696 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:45.678468 (+ 772us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:45.678749 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:45.680172 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:45.681287 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:45.681881 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:45.681994 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:45.684997 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:45.685002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:45.685002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:45.685016 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:45.687898 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:45.688405 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:45.688409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:45.688409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:45.688539 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:45.688810 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:45.688811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:45.688811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:45.689168 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:45.689352 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:45.689462 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 07:23:45.689579 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260810 07:23:45.690922 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:45.691741 14369 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:45.692368 14369 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:45.696774 14369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e58563943f4476bbf26fafa5cd3d495: Generated new TSK 0
I20260810 07:23:45.697006 14369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:45.701257 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:45.692994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55863 (local address 127.0.0.1:46819)
0810 07:23:45.693054 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:45.693056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:45.693314 (+ 258us) server_negotiation.cc:407] Connection header received
0810 07:23:45.693378 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:45.693379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:45.693389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:45.693443 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:45.694090 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:45.694849 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:45.695940 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:45.696228 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:45.696301 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:45.697986 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:45.697987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:45.697988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:45.697996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:45.699841 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:45.700300 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:45.700301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:45.700301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:45.700369 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:45.700619 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:45.700620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:45.700620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:45.700818 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:45.700920 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:45.701075 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 07:23:45.701136 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:23:45.703619 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 14239
W20260810 07:23:45.716975 14213 connection.cc:535] client connection to 127.0.0.1:46819 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46819): Connection reset by peer (error 104)
2026-08-10T07:23:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:45 dist-test-slave-x5fp krb5kdc[14396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:45 dist-test-slave-x5fp krb5kdc[14396](info): set up 2 sockets
Aug 10 07:23:45 dist-test-slave-x5fp krb5kdc[14396](info): commencing operation
krb5kdc: starting...
W20260810 07:23:47.037165 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:47 dist-test-slave-x5fp krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:47.181175 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:47.268169 14415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:47.268419 14415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:47.268458 14415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:47.269676 14415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:47.269753 14415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:47.269778 14415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:47.269809 14415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:47.269840 14415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:47.271332 14415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:47.271615 14415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:47.271870 14415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:47.277235 14427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:47.277248 14424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:47.277248 14429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:47.277428 14415 server_base.cc:1034] running on GCE node
I20260810 07:23:47.277801 14415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:47.278121 14415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:47.279302 14415 hybrid_clock.cc:648] HybridClock initialized: now 1786346627279279 us; error 44 us; skew 500 ppm
Aug 10 07:23:47 dist-test-slave-x5fp krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346627, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:47.282135 14415 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:47.282663 14415 webserver.cc:460] Webserver started at http://127.0.0.1:46191/ using document root <none> and password file <none>
I20260810 07:23:47.282981 14415 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:47.283059 14415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:47.283218 14415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:47.284359 14415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "fcceee168513473c8ee82c03b231fb64"
format_stamp: "Formatted at 2026-08-10 07:23:47 on dist-test-slave-x5fp"
server_key: "90d2c9318f282611aa7b20476d573b1e"
server_key_iv: "b937e127a9f6b9cdb802fed48afab4fc"
server_key_version: "encryptionkey@0"
I20260810 07:23:47.284854 14415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "fcceee168513473c8ee82c03b231fb64"
format_stamp: "Formatted at 2026-08-10 07:23:47 on dist-test-slave-x5fp"
server_key: "90d2c9318f282611aa7b20476d573b1e"
server_key_iv: "b937e127a9f6b9cdb802fed48afab4fc"
server_key_version: "encryptionkey@0"
I20260810 07:23:47.286273 14415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:47.287170 14439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:47.287437 14415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:47.287544 14415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "fcceee168513473c8ee82c03b231fb64"
format_stamp: "Formatted at 2026-08-10 07:23:47 on dist-test-slave-x5fp"
server_key: "90d2c9318f282611aa7b20476d573b1e"
server_key_iv: "b937e127a9f6b9cdb802fed48afab4fc"
server_key_version: "encryptionkey@0"
I20260810 07:23:47.287606 14415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:47.307837 14415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:47.310415 14415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:47.310541 14415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:47.314772 14415 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41459
I20260810 07:23:47.314805 14523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41459 every 8 connection(s)
I20260810 07:23:47.315179 14415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:47.316076 14524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:47.317996 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 14415
I20260810 07:23:47.318143 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:47.318363 14992 external_mini_cluster.cc:1428] Setting key baf8e31ba5020c3b80510a6d477d1134
I20260810 07:23:47.318696 14524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64: Bootstrap starting.
I20260810 07:23:47.319495 14524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:47.319837 14524 log.cc:824] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:47.320895 14524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64: No bootstrap required, opened a new log
I20260810 07:23:47.323547 14524 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcceee168513473c8ee82c03b231fb64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41459 } }
I20260810 07:23:47.323752 14524 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:47.323797 14524 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcceee168513473c8ee82c03b231fb64, State: Initialized, Role: FOLLOWER
I20260810 07:23:47.323973 14524 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcceee168513473c8ee82c03b231fb64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41459 } }
I20260810 07:23:47.324081 14524 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:47.324165 14524 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:47.324231 14524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:47.325057 14524 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcceee168513473c8ee82c03b231fb64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41459 } }
I20260810 07:23:47.325186 14524 leader_election.cc:302] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcceee168513473c8ee82c03b231fb64; no voters:
I20260810 07:23:47.325412 14524 leader_election.cc:288] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:47.325613 14532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:47.325680 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.318974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59499 (local address 127.0.0.1:41459)
0810 07:23:47.319191 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:47.319203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:47.319240 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:23:47.319936 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:47.319945 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:47.320016 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:47.320166 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:47.320523 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:47.322201 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:47.323100 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:47.323828 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:47.323963 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:47.324840 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:47.324845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:47.324846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:47.325091 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:47.325139 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:47.325157 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:47.325263 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:47.325362 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 07:23:47.325395 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260810 07:23:47.325743 14524 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:47.325783 14532 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [term 1 LEADER]: Becoming Leader. State: Replica: fcceee168513473c8ee82c03b231fb64, State: Running, Role: LEADER
I20260810 07:23:47.326707 14532 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcceee168513473c8ee82c03b231fb64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41459 } }
I20260810 07:23:47.326974 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:47.327386 14534 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcceee168513473c8ee82c03b231fb64. Latest consensus state: current_term: 1 leader_uuid: "fcceee168513473c8ee82c03b231fb64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcceee168513473c8ee82c03b231fb64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41459 } } }
I20260810 07:23:47.327543 14530 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcceee168513473c8ee82c03b231fb64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcceee168513473c8ee82c03b231fb64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41459 } } }
I20260810 07:23:47.327694 14534 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:47.327695 14530 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64 [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:47.328569 14551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:47.328691 14551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:47.328892 14552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:47.329092 14552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:47.330830 14552 catalog_manager.cc:1347] Generated new cluster ID: 67b48269fa134c3a8c04672baac3403f
I20260810 07:23:47.330897 14552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:47.333179 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.328673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50403 (local address 127.0.0.1:41459)
0810 07:23:47.328710 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:47.328733 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:47.329878 (+ 1145us) server_negotiation.cc:407] Connection header received
0810 07:23:47.329940 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:47.329941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:47.329951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:47.330030 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:47.330230 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:47.330962 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:47.331749 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:47.331998 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:47.332075 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:47.332908 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:47.332910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:47.332910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:47.332914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:47.332930 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:47.332934 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:47.332993 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:47.333067 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 07:23:47.333079 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:23:47.349902 14552 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:47.350505 14552 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:47.356403 14552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcceee168513473c8ee82c03b231fb64: Generated new TSK 0
I20260810 07:23:47.356680 14552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:47.363564 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 14415
W20260810 07:23:47.374790 14390 connection.cc:535] client connection to 127.0.0.1:41459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41459): Connection reset by peer (error 104)
2026-08-10T07:23:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1657 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:47 dist-test-slave-x5fp krb5kdc[14574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:47 dist-test-slave-x5fp krb5kdc[14574](info): set up 2 sockets
Aug 10 07:23:47 dist-test-slave-x5fp krb5kdc[14574](info): commencing operation
krb5kdc: starting...
W20260810 07:23:48.664572 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:48 dist-test-slave-x5fp krb5kdc[14574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:48.797879 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:48.887005 14590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:48.887300 14590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:48.887343 14590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:48.888553 14590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:48.888636 14590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:48.888665 14590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:48.888711 14590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:48.888803 14590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:48.890372 14590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:48.890707 14590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:48.890982 14590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:48.896453 14605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:48.896589 14590 server_base.cc:1034] running on GCE node
W20260810 07:23:48.896394 14601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:48.896423 14608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:48.897007 14590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:48.897293 14590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:48.898455 14590 hybrid_clock.cc:648] HybridClock initialized: now 1786346628898440 us; error 29 us; skew 500 ppm
Aug 10 07:23:48 dist-test-slave-x5fp krb5kdc[14574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:48.901114 14590 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:48.901616 14590 webserver.cc:460] Webserver started at http://127.0.0.1:42231/ using document root <none> and password file <none>
I20260810 07:23:48.901866 14590 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:48.901940 14590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:48.902104 14590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:48.903216 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "668cdcd1d3b2454f9edf4915a022cd48"
format_stamp: "Formatted at 2026-08-10 07:23:48 on dist-test-slave-x5fp"
server_key: "50fb9468e085e6607e3f76de62c1a3a9"
server_key_iv: "e72207f54a3e27e9c2017d400c9db96b"
server_key_version: "encryptionkey@0"
I20260810 07:23:48.903609 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "668cdcd1d3b2454f9edf4915a022cd48"
format_stamp: "Formatted at 2026-08-10 07:23:48 on dist-test-slave-x5fp"
server_key: "50fb9468e085e6607e3f76de62c1a3a9"
server_key_iv: "e72207f54a3e27e9c2017d400c9db96b"
server_key_version: "encryptionkey@0"
I20260810 07:23:48.905126 14590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:23:48.906005 14617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:48.906225 14590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:48.906344 14590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "668cdcd1d3b2454f9edf4915a022cd48"
format_stamp: "Formatted at 2026-08-10 07:23:48 on dist-test-slave-x5fp"
server_key: "50fb9468e085e6607e3f76de62c1a3a9"
server_key_iv: "e72207f54a3e27e9c2017d400c9db96b"
server_key_version: "encryptionkey@0"
I20260810 07:23:48.906430 14590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:48.909948 14590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:48.912038 14590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:48.912201 14590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:48.916326 14590 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39323
I20260810 07:23:48.916383 14699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39323 every 8 connection(s)
I20260810 07:23:48.916831 14590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:48.918011 14700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:48.920348 14700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48: Bootstrap starting.
I20260810 07:23:48.921007 14700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:48.921263 14700 log.cc:824] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:48.922082 14700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48: No bootstrap required, opened a new log
I20260810 07:23:48.923858 14700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668cdcd1d3b2454f9edf4915a022cd48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260810 07:23:48.924009 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 14590
I20260810 07:23:48.924069 14700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:48.924118 14700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cdcd1d3b2454f9edf4915a022cd48, State: Initialized, Role: FOLLOWER
I20260810 07:23:48.924139 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:48.924295 14700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668cdcd1d3b2454f9edf4915a022cd48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260810 07:23:48.924389 14700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:48.924391 14992 external_mini_cluster.cc:1428] Setting key 7ad1be42caafcc4a54155cf448eb8983
I20260810 07:23:48.924448 14700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:48.924510 14700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:48.925379 14700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668cdcd1d3b2454f9edf4915a022cd48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260810 07:23:48.925529 14700 leader_election.cc:302] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 668cdcd1d3b2454f9edf4915a022cd48; no voters:
I20260810 07:23:48.925716 14700 leader_election.cc:288] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:48.925781 14709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:48.925987 14700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:48.925987 14709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [term 1 LEADER]: Becoming Leader. State: Replica: 668cdcd1d3b2454f9edf4915a022cd48, State: Running, Role: LEADER
I20260810 07:23:48.926258 14709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668cdcd1d3b2454f9edf4915a022cd48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260810 07:23:48.926679 14711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "668cdcd1d3b2454f9edf4915a022cd48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668cdcd1d3b2454f9edf4915a022cd48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } } }
I20260810 07:23:48.926694 14712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 668cdcd1d3b2454f9edf4915a022cd48. Latest consensus state: current_term: 1 leader_uuid: "668cdcd1d3b2454f9edf4915a022cd48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668cdcd1d3b2454f9edf4915a022cd48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } } }
I20260810 07:23:48.926785 14711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:48.926807 14712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:48.927111 14714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:48.927287 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.924984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35019 (local address 127.0.0.1:39323)
0810 07:23:48.925505 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:48.925515 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:48.925555 (+ 40us) server_negotiation.cc:407] Connection header received
0810 07:23:48.925685 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:48.925693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:48.925725 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:48.925882 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:48.926660 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:48.926664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:48.926665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:48.926670 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:48.926719 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:48.926735 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:48.926797 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:48.926990 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:23:48.927038 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":393,"threads_started":1}
I20260810 07:23:48.927654 14714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:48.928373 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:48.929415 14714 catalog_manager.cc:1347] Generated new cluster ID: 6fc63af9baf44590855b8612c0c85c45
I20260810 07:23:48.929471 14714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:48.931280 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.930141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50347 (local address 127.0.0.1:39323)
0810 07:23:48.930177 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:48.930179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:48.930283 (+ 104us) server_negotiation.cc:407] Connection header received
0810 07:23:48.930315 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:48.930316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:48.930321 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:48.930402 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:48.931037 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:48.931039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:48.931039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:48.931044 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:48.931062 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:48.931067 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:48.931137 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:48.931197 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 07:23:48.931211 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 07:23:48.945619 14714 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:48.945813 14714 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:48.954217 14714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 668cdcd1d3b2454f9edf4915a022cd48: Generated new TSK 0
I20260810 07:23:48.954439 14714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:48.956327 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 14590
2026-08-10T07:23:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:49 dist-test-slave-x5fp krb5kdc[14747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:49 dist-test-slave-x5fp krb5kdc[14747](info): set up 2 sockets
Aug 10 07:23:49 dist-test-slave-x5fp krb5kdc[14747](info): commencing operation
krb5kdc: starting...
W20260810 07:23:50.217852 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:50 dist-test-slave-x5fp krb5kdc[14747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:50.353178 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:50.439318 14764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:50.439546 14764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:50.439569 14764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:50.440799 14764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:50.440845 14764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:50.440860 14764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:50.440876 14764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:50.440887 14764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:50.442440 14764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:50.442708 14764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:50.442950 14764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:50.447932 14774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:50.447969 14773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:50.447979 14777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:50.448133 14764 server_base.cc:1034] running on GCE node
I20260810 07:23:50.448468 14764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:50.448827 14764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:50.450003 14764 hybrid_clock.cc:648] HybridClock initialized: now 1786346630449981 us; error 40 us; skew 500 ppm
Aug 10 07:23:50 dist-test-slave-x5fp krb5kdc[14747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:50.452662 14764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:50.453193 14764 webserver.cc:460] Webserver started at http://127.0.0.1:40589/ using document root <none> and password file <none>
I20260810 07:23:50.453454 14764 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:50.453527 14764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:50.453679 14764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:50.454751 14764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "0396107030f846b4b5359745b67829ce"
format_stamp: "Formatted at 2026-08-10 07:23:50 on dist-test-slave-x5fp"
server_key: "2fa9f6f485e71bdd7dfbe4cc0cec56ec"
server_key_iv: "c363f6b8d9a28f8732a7b6286182c0df"
server_key_version: "encryptionkey@0"
I20260810 07:23:50.455133 14764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "0396107030f846b4b5359745b67829ce"
format_stamp: "Formatted at 2026-08-10 07:23:50 on dist-test-slave-x5fp"
server_key: "2fa9f6f485e71bdd7dfbe4cc0cec56ec"
server_key_iv: "c363f6b8d9a28f8732a7b6286182c0df"
server_key_version: "encryptionkey@0"
I20260810 07:23:50.456487 14764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:23:50.457443 14792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:50.457679 14764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:50.457804 14764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "0396107030f846b4b5359745b67829ce"
format_stamp: "Formatted at 2026-08-10 07:23:50 on dist-test-slave-x5fp"
server_key: "2fa9f6f485e71bdd7dfbe4cc0cec56ec"
server_key_iv: "c363f6b8d9a28f8732a7b6286182c0df"
server_key_version: "encryptionkey@0"
I20260810 07:23:50.457899 14764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:50.476338 14764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:50.478855 14764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:50.479038 14764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:50.483471 14764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43539
I20260810 07:23:50.483532 14878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43539 every 8 connection(s)
I20260810 07:23:50.483922 14764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:50.484844 14879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:50.487156 14879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce: Bootstrap starting.
I20260810 07:23:50.487764 14879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:50.488018 14879 log.cc:824] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:50.488703 14879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce: No bootstrap required, opened a new log
I20260810 07:23:50.489760 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 14764
I20260810 07:23:50.489923 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:50.490124 14992 external_mini_cluster.cc:1428] Setting key 0583dcdeafcd31f757d1cee626c67cc6
I20260810 07:23:50.490625 14879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0396107030f846b4b5359745b67829ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260810 07:23:50.490844 14879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:50.490911 14879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0396107030f846b4b5359745b67829ce, State: Initialized, Role: FOLLOWER
I20260810 07:23:50.491050 14879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0396107030f846b4b5359745b67829ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260810 07:23:50.491134 14879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:50.491165 14879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:50.491214 14879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:50.492270 14879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0396107030f846b4b5359745b67829ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260810 07:23:50.492385 14879 leader_election.cc:302] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0396107030f846b4b5359745b67829ce; no voters:
I20260810 07:23:50.492553 14879 leader_election.cc:288] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:50.492612 14886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:50.492834 14879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:50.492833 14886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [term 1 LEADER]: Becoming Leader. State: Replica: 0396107030f846b4b5359745b67829ce, State: Running, Role: LEADER
I20260810 07:23:50.493032 14886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0396107030f846b4b5359745b67829ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260810 07:23:50.493412 14886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0396107030f846b4b5359745b67829ce. Latest consensus state: current_term: 1 leader_uuid: "0396107030f846b4b5359745b67829ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0396107030f846b4b5359745b67829ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } } }
I20260810 07:23:50.493656 14886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:50.493566 14887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0396107030f846b4b5359745b67829ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0396107030f846b4b5359745b67829ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } } }
I20260810 07:23:50.493916 14887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:50.494319 14900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:50.495107 14900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:50.496842 14900 catalog_manager.cc:1347] Generated new cluster ID: d9958727317844ac9059f9c3017fe356
I20260810 07:23:50.496894 14900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:23:50 dist-test-slave-x5fp krb5kdc[14747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:23:50.501724 14885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:50.490705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57017 (local address 127.0.0.1:43539)
0810 07:23:50.490985 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:50.490995 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:50.491030 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:23:50.491148 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:50.491156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:50.491213 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:50.491373 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:50.492010 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:50.493520 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:50.494735 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:50.495195 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:50.495312 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:50.498322 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:50.498327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:50.498327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:50.498340 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:50.500076 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:50.500466 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:50.500469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:50.500470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:50.500587 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:50.500809 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:50.500810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:50.500810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:50.501052 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:50.501239 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:50.501308 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:23:50.501571 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":122,"threads_started":1}
I20260810 07:23:50.502679 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:50.509721 14900 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:50.510408 14900 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:50.510548 14885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:50.504107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39569 (local address 127.0.0.1:43539)
0810 07:23:50.504149 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:50.504150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:50.504278 (+ 128us) server_negotiation.cc:407] Connection header received
0810 07:23:50.504351 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:50.504352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:50.504363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:50.504411 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:50.504980 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:50.505479 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:50.506192 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:50.506414 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:50.506492 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:50.507678 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:50.507679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:50.507680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:50.507686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:50.509283 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:50.509708 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:50.509709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:50.509709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:50.509760 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:50.510007 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:50.510008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:50.510009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:50.510125 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:50.510260 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:50.510344 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:23:50.510429 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:23:50.515551 14900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0396107030f846b4b5359745b67829ce: Generated new TSK 0
I20260810 07:23:50.515758 14900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:50.518085 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 14764
2026-08-10T07:23:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:50 dist-test-slave-x5fp krb5kdc[14925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:50 dist-test-slave-x5fp krb5kdc[14925](info): set up 2 sockets
Aug 10 07:23:50 dist-test-slave-x5fp krb5kdc[14925](info): commencing operation
krb5kdc: starting...
W20260810 07:23:51.857082 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:51 dist-test-slave-x5fp krb5kdc[14925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:52.000468 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:52.085724 14944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:52.085961 14944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:52.085986 14944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:52.087230 14944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:52.087283 14944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:52.087298 14944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:52.087311 14944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:52.087323 14944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:52.088982 14944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:52.089264 14944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:52.089533 14944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:52.094919 14953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:52.095175 14944 server_base.cc:1034] running on GCE node
W20260810 07:23:52.094919 14956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:52.094916 14951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:52.095643 14944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:52.096065 14944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:52.097268 14944 hybrid_clock.cc:648] HybridClock initialized: now 1786346632097243 us; error 44 us; skew 500 ppm
Aug 10 07:23:52 dist-test-slave-x5fp krb5kdc[14925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:52.100116 14944 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:52.100652 14944 webserver.cc:460] Webserver started at http://127.0.0.1:37387/ using document root <none> and password file <none>
I20260810 07:23:52.100937 14944 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:52.101015 14944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:52.101182 14944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:52.102459 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "d23014256211493c949236b66ea7d5ab"
format_stamp: "Formatted at 2026-08-10 07:23:52 on dist-test-slave-x5fp"
server_key: "af2f72de2e01caf3ffcd7e235cb7323d"
server_key_iv: "784ded22065ef45d6997cf7459f606df"
server_key_version: "encryptionkey@0"
I20260810 07:23:52.102859 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "d23014256211493c949236b66ea7d5ab"
format_stamp: "Formatted at 2026-08-10 07:23:52 on dist-test-slave-x5fp"
server_key: "af2f72de2e01caf3ffcd7e235cb7323d"
server_key_iv: "784ded22065ef45d6997cf7459f606df"
server_key_version: "encryptionkey@0"
I20260810 07:23:52.104387 14944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:52.105310 14966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:52.105561 14944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:52.105686 14944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "d23014256211493c949236b66ea7d5ab"
format_stamp: "Formatted at 2026-08-10 07:23:52 on dist-test-slave-x5fp"
server_key: "af2f72de2e01caf3ffcd7e235cb7323d"
server_key_iv: "784ded22065ef45d6997cf7459f606df"
server_key_version: "encryptionkey@0"
I20260810 07:23:52.105769 14944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:52.125032 14944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:52.127302 14944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:52.127487 14944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:52.131913 14944 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38695
I20260810 07:23:52.132038 15049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38695 every 8 connection(s)
I20260810 07:23:52.132375 14944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:52.133422 15050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:52.135726 15050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab: Bootstrap starting.
I20260810 07:23:52.136270 15050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:52.136520 15050 log.cc:824] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:52.136937 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 14944
I20260810 07:23:52.137100 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:52.137297 14992 external_mini_cluster.cc:1428] Setting key 850558f4042be0d9d5e75409769d1817
I20260810 07:23:52.137456 15050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab: No bootstrap required, opened a new log
I20260810 07:23:52.139401 15050 raft_consensus.cc:348] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d23014256211493c949236b66ea7d5ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260810 07:23:52.139568 15050 raft_consensus.cc:374] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:52.139636 15050 raft_consensus.cc:729] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23014256211493c949236b66ea7d5ab, State: Initialized, Role: FOLLOWER
I20260810 07:23:52.139782 15050 consensus_queue.cc:260] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d23014256211493c949236b66ea7d5ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260810 07:23:52.139900 15050 raft_consensus.cc:388] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:52.139950 15050 raft_consensus.cc:482] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:52.140002 15050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:52.140578 15050 raft_consensus.cc:504] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d23014256211493c949236b66ea7d5ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260810 07:23:52.140702 15050 leader_election.cc:302] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d23014256211493c949236b66ea7d5ab; no voters:
I20260810 07:23:52.140921 15050 leader_election.cc:288] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:52.140977 15063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:52.141134 15063 raft_consensus.cc:686] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [term 1 LEADER]: Becoming Leader. State: Replica: d23014256211493c949236b66ea7d5ab, State: Running, Role: LEADER
I20260810 07:23:52.141196 15050 sys_catalog.cc:564] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:52.141317 15063 consensus_queue.cc:237] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d23014256211493c949236b66ea7d5ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260810 07:23:52.141701 15063 sys_catalog.cc:455] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader d23014256211493c949236b66ea7d5ab. Latest consensus state: current_term: 1 leader_uuid: "d23014256211493c949236b66ea7d5ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d23014256211493c949236b66ea7d5ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260810 07:23:52.141840 15063 sys_catalog.cc:458] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:52.142098 15079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:52.142928 15079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:52.142997 15065 sys_catalog.cc:455] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d23014256211493c949236b66ea7d5ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d23014256211493c949236b66ea7d5ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260810 07:23:52.143141 15065 sys_catalog.cc:458] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:52.144783 15079 catalog_manager.cc:1347] Generated new cluster ID: 66a0d1e46b58475fb1e441f60d4d9390
I20260810 07:23:52.144836 15079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:52.145115 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:52.137872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52947 (local address 127.0.0.1:38695)
0810 07:23:52.138048 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:52.138057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:52.138092 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:23:52.139401 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:52.139409 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:52.139474 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:52.139652 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:52.139909 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:52.141531 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:52.142776 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:52.143404 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:52.143538 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:52.144364 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:52.144368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:52.144368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:52.144576 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:52.144609 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:52.144621 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:52.144704 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:52.144802 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 07:23:52.144874 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260810 07:23:52.149276 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:52.153959 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:52.150859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55405 (local address 127.0.0.1:38695)
0810 07:23:52.150905 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:52.150907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:52.151012 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:23:52.151092 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:52.151093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:52.151104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:52.151160 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:52.151338 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:52.151864 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:52.152618 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:52.152909 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:52.152987 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:52.153712 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:52.153713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:52.153713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:52.153718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:52.153733 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:52.153738 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:52.153785 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:52.153837 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:23:52.153861 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:23:52.198391 15079 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:52.199186 15079 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:52.210067 15079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d23014256211493c949236b66ea7d5ab: Generated new TSK 0
I20260810 07:23:52.210299 15079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:52.225355 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 14944
2026-08-10T07:23:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1703 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:52 dist-test-slave-x5fp krb5kdc[15107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:52 dist-test-slave-x5fp krb5kdc[15107](info): set up 2 sockets
Aug 10 07:23:52 dist-test-slave-x5fp krb5kdc[15107](info): commencing operation
krb5kdc: starting...
W20260810 07:23:53.508821 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:53 dist-test-slave-x5fp krb5kdc[15107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:53.639763 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33683
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:53.727757 15124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:53.728067 15124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:53.728118 15124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:53.729457 15124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:53.729545 15124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:53.729574 15124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:53.729606 15124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:53.729640 15124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:53.731309 15124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:53.731628 15124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:53.731904 15124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:53.737387 15132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:53.737380 15130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:53.737383 15135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:53.737788 15124 server_base.cc:1034] running on GCE node
I20260810 07:23:53.738058 15124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:53.738361 15124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:53.739540 15124 hybrid_clock.cc:648] HybridClock initialized: now 1786346633739524 us; error 28 us; skew 500 ppm
Aug 10 07:23:53 dist-test-slave-x5fp krb5kdc[15107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346633, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:53.742268 15124 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:53.742784 15124 webserver.cc:460] Webserver started at http://127.0.0.1:41477/ using document root <none> and password file <none>
I20260810 07:23:53.743027 15124 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:53.743098 15124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:53.743258 15124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:53.744415 15124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "9b056a1f96c44a7ba224bd900581ce2c"
format_stamp: "Formatted at 2026-08-10 07:23:53 on dist-test-slave-x5fp"
server_key: "dbf274248e0a7e78615d5eba8c9fe9d5"
server_key_iv: "70f53750da409ab1ba6de24848bf2931"
server_key_version: "encryptionkey@0"
I20260810 07:23:53.744882 15124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "9b056a1f96c44a7ba224bd900581ce2c"
format_stamp: "Formatted at 2026-08-10 07:23:53 on dist-test-slave-x5fp"
server_key: "dbf274248e0a7e78615d5eba8c9fe9d5"
server_key_iv: "70f53750da409ab1ba6de24848bf2931"
server_key_version: "encryptionkey@0"
I20260810 07:23:53.746362 15124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:53.747221 15146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:53.747555 15124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:53.747674 15124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "9b056a1f96c44a7ba224bd900581ce2c"
format_stamp: "Formatted at 2026-08-10 07:23:53 on dist-test-slave-x5fp"
server_key: "dbf274248e0a7e78615d5eba8c9fe9d5"
server_key_iv: "70f53750da409ab1ba6de24848bf2931"
server_key_version: "encryptionkey@0"
I20260810 07:23:53.747761 15124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:53.751307 15124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:53.753881 15124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:53.754053 15124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:53.758359 15124 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33683
I20260810 07:23:53.758415 15233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33683 every 8 connection(s)
I20260810 07:23:53.758792 15124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:53.759915 15234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:53.762374 15234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c: Bootstrap starting.
I20260810 07:23:53.762943 15234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:53.763182 15234 log.cc:824] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:53.763911 15234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c: No bootstrap required, opened a new log
I20260810 07:23:53.765586 15234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b056a1f96c44a7ba224bd900581ce2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33683 } }
I20260810 07:23:53.765794 15234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:53.765851 15234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b056a1f96c44a7ba224bd900581ce2c, State: Initialized, Role: FOLLOWER
I20260810 07:23:53.765867 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15124
I20260810 07:23:53.765980 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:53.766014 15234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b056a1f96c44a7ba224bd900581ce2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33683 } }
I20260810 07:23:53.766110 15234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:53.766145 15234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:53.766184 14992 external_mini_cluster.cc:1428] Setting key f1d85e0ea42054524b777490a6b5c3ff
I20260810 07:23:53.766197 15234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:53.766917 15234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b056a1f96c44a7ba224bd900581ce2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33683 } }
I20260810 07:23:53.767057 15234 leader_election.cc:302] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b056a1f96c44a7ba224bd900581ce2c; no voters:
I20260810 07:23:53.767230 15234 leader_election.cc:288] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:53.767261 15241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:53.767405 15241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [term 1 LEADER]: Becoming Leader. State: Replica: 9b056a1f96c44a7ba224bd900581ce2c, State: Running, Role: LEADER
I20260810 07:23:53.767530 15234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:53.767599 15241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b056a1f96c44a7ba224bd900581ce2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33683 } }
I20260810 07:23:53.768900 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:53.766734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54747 (local address 127.0.0.1:33683)
0810 07:23:53.766943 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:53.766951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:53.766979 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:23:53.767138 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:53.767146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:53.767178 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:53.767327 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:53.768251 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:53.768255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:53.768255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:53.768260 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:53.768332 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:53.768350 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:53.768460 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:53.768594 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 07:23:53.768639 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260810 07:23:53.769016 15247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b056a1f96c44a7ba224bd900581ce2c. Latest consensus state: current_term: 1 leader_uuid: "9b056a1f96c44a7ba224bd900581ce2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b056a1f96c44a7ba224bd900581ce2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33683 } } }
I20260810 07:23:53.769425 15247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:53.768857 15245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b056a1f96c44a7ba224bd900581ce2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b056a1f96c44a7ba224bd900581ce2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33683 } } }
I20260810 07:23:53.769587 15245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:53.769600 15265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:53.769663 15265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:53.770262 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:53.770617 15268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:53.770781 15268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:53.772501 15268 catalog_manager.cc:1347] Generated new cluster ID: 1d45b2e8ed3d4463ad7f69dc950f1c29
I20260810 07:23:53.772553 15268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:53.773262 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:53.771923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48567 (local address 127.0.0.1:33683)
0810 07:23:53.771957 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:53.771959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:53.772151 (+ 192us) server_negotiation.cc:407] Connection header received
0810 07:23:53.772199 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:53.772200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:53.772205 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:53.772275 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:53.772978 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:53.772979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:53.772980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:53.772984 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:53.772999 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:53.773004 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:53.773050 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:53.773113 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:23:53.773128 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:23:53.802238 15268 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:53.802471 15268 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:53.806794 15268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b056a1f96c44a7ba224bd900581ce2c: Generated new TSK 0
I20260810 07:23:53.807015 15268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:53.817058 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15124
2026-08-10T07:23:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1595 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:53 dist-test-slave-x5fp krb5kdc[15283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:53 dist-test-slave-x5fp krb5kdc[15283](info): set up 2 sockets
Aug 10 07:23:53 dist-test-slave-x5fp krb5kdc[15283](info): commencing operation
krb5kdc: starting...
W20260810 07:23:55.090775 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:55 dist-test-slave-x5fp krb5kdc[15283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:55.224916 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45989
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:55.310698 15301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:55.310961 15301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:55.310999 15301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:55.312173 15301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:55.312268 15301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:55.312297 15301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:55.312328 15301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:55.312352 15301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:55.313936 15301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:55.314241 15301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:55.314493 15301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:55.319727 15321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:55.319777 15314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:55.320006 15301 server_base.cc:1034] running on GCE node
W20260810 07:23:55.319808 15316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:55.320457 15301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:55.320851 15301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:55.322057 15301 hybrid_clock.cc:648] HybridClock initialized: now 1786346635322034 us; error 53 us; skew 500 ppm
Aug 10 07:23:55 dist-test-slave-x5fp krb5kdc[15283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:55.324896 15301 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:55.325412 15301 webserver.cc:460] Webserver started at http://127.0.0.1:38177/ using document root <none> and password file <none>
I20260810 07:23:55.325672 15301 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:55.325742 15301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:55.325898 15301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:55.327019 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "126931bb99d6495688d4f0f2b439b470"
format_stamp: "Formatted at 2026-08-10 07:23:55 on dist-test-slave-x5fp"
server_key: "07a94c4ba749344790465d04c84c1bad"
server_key_iv: "37556105c74606631ae8b6366e471b52"
server_key_version: "encryptionkey@0"
I20260810 07:23:55.327557 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "126931bb99d6495688d4f0f2b439b470"
format_stamp: "Formatted at 2026-08-10 07:23:55 on dist-test-slave-x5fp"
server_key: "07a94c4ba749344790465d04c84c1bad"
server_key_iv: "37556105c74606631ae8b6366e471b52"
server_key_version: "encryptionkey@0"
I20260810 07:23:55.329141 15301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:55.330086 15330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:55.330356 15301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:55.330489 15301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "126931bb99d6495688d4f0f2b439b470"
format_stamp: "Formatted at 2026-08-10 07:23:55 on dist-test-slave-x5fp"
server_key: "07a94c4ba749344790465d04c84c1bad"
server_key_iv: "37556105c74606631ae8b6366e471b52"
server_key_version: "encryptionkey@0"
I20260810 07:23:55.330577 15301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:55.357877 15301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:55.360561 15301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:55.360796 15301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:55.365242 15301 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45989
I20260810 07:23:55.365298 15410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45989 every 8 connection(s)
I20260810 07:23:55.365702 15301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:55.366703 15411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:55.369155 15411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470: Bootstrap starting.
I20260810 07:23:55.369736 15411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:55.369988 15411 log.cc:824] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:55.370767 15411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470: No bootstrap required, opened a new log
I20260810 07:23:55.371678 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15301
I20260810 07:23:55.371799 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:55.372009 14992 external_mini_cluster.cc:1428] Setting key 2d8366618d631e6dba6c772ee2663187
I20260810 07:23:55.373181 15411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126931bb99d6495688d4f0f2b439b470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45989 } }
I20260810 07:23:55.373368 15411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:55.373409 15411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 126931bb99d6495688d4f0f2b439b470, State: Initialized, Role: FOLLOWER
I20260810 07:23:55.373589 15411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126931bb99d6495688d4f0f2b439b470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45989 } }
I20260810 07:23:55.373695 15411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:55.373733 15411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:55.373793 15411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:55.374871 15411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126931bb99d6495688d4f0f2b439b470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45989 } }
I20260810 07:23:55.374970 15411 leader_election.cc:302] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 126931bb99d6495688d4f0f2b439b470; no voters:
I20260810 07:23:55.375162 15411 leader_election.cc:288] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:55.375223 15419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:55.375391 15419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [term 1 LEADER]: Becoming Leader. State: Replica: 126931bb99d6495688d4f0f2b439b470, State: Running, Role: LEADER
I20260810 07:23:55.375458 15411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:55.375607 15419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126931bb99d6495688d4f0f2b439b470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45989 } }
I20260810 07:23:55.376948 15421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 126931bb99d6495688d4f0f2b439b470. Latest consensus state: current_term: 1 leader_uuid: "126931bb99d6495688d4f0f2b439b470" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126931bb99d6495688d4f0f2b439b470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45989 } } }
I20260810 07:23:55.377178 15420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "126931bb99d6495688d4f0f2b439b470" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126931bb99d6495688d4f0f2b439b470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45989 } } }
I20260810 07:23:55.377338 15421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:55.377352 15420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470 [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:55.377660 15436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:55.377810 15436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:55.377921 15438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:55.378154 15438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:55.379967 15438 catalog_manager.cc:1347] Generated new cluster ID: 835277f1fe514529a6ac9f1906d61b28
I20260810 07:23:55.380031 15438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:23:55 dist-test-slave-x5fp krb5kdc[15283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:23:55.385514 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:55.372768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:51741 (local address 127.0.0.1:45989)
0810 07:23:55.373015 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:55.373025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:55.373056 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:23:55.373174 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:55.373180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:55.373233 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:55.373385 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:55.374241 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:55.375361 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:55.376563 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:55.377500 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:55.377664 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:55.381001 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:55.381006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:55.381006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:55.381020 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:55.383209 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:55.383778 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:55.383781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:55.383782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:55.383951 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:55.384240 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:55.384241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:55.384241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:55.384586 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:55.384926 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:55.385147 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:23:55.385283 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":126,"threads_started":1}
I20260810 07:23:55.387004 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:55.390384 15438 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:55.391139 15438 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:55.396276 15438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 126931bb99d6495688d4f0f2b439b470: Generated new TSK 0
I20260810 07:23:55.396533 15438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:55.402174 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:55.388693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:58307 (local address 127.0.0.1:45989)
0810 07:23:55.388775 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:55.388777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:55.393050 (+ 4273us) server_negotiation.cc:407] Connection header received
0810 07:23:55.393379 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:55.393381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:55.393399 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:55.393493 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:55.394333 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:55.394899 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:55.396002 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:55.396379 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:55.396496 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:55.398580 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:55.398583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:55.398583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:55.398596 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:55.400566 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:55.401056 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:55.401058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:55.401058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:55.401130 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:55.401408 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:55.401409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:55.401410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:55.401541 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:55.401665 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:55.401826 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 07:23:55.401905 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:23:55.405092 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15301
W20260810 07:23:55.435384 15281 connection.cc:535] client connection to 127.0.0.1:45989 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45989): Connection reset by peer (error 104)
2026-08-10T07:23:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:55 dist-test-slave-x5fp krb5kdc[15452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:55 dist-test-slave-x5fp krb5kdc[15452](info): set up 2 sockets
Aug 10 07:23:55 dist-test-slave-x5fp krb5kdc[15452](info): commencing operation
krb5kdc: starting...
W20260810 07:23:56.726456 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:56 dist-test-slave-x5fp krb5kdc[15452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:56.868007 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33157
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:56.955500 15470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:56.955837 15470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:56.955881 15470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:56.958003 15470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:56.958086 15470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:56.958115 15470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:56.958140 15470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:56.958225 15470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:56.960878 15470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:56.961221 15470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:56.961529 15470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:56.968235 15479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:56.968350 15478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:56.968350 15482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:56.968957 15470 server_base.cc:1034] running on GCE node
I20260810 07:23:56.969327 15470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:56.969664 15470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:56.970839 15470 hybrid_clock.cc:648] HybridClock initialized: now 1786346636970806 us; error 45 us; skew 500 ppm
Aug 10 07:23:56 dist-test-slave-x5fp krb5kdc[15452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:56.974717 15470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:56.975359 15470 webserver.cc:460] Webserver started at http://127.0.0.1:40755/ using document root <none> and password file <none>
I20260810 07:23:56.975647 15470 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:56.975726 15470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:56.975890 15470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:56.977725 15470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "e5f3c854f5a1404ebea932673f1ed717"
format_stamp: "Formatted at 2026-08-10 07:23:56 on dist-test-slave-x5fp"
server_key: "0c8bdd3b80185df472839dc72939fb59"
server_key_iv: "7bc3fa9b7376524b9becf429c9d1b82a"
server_key_version: "encryptionkey@0"
I20260810 07:23:56.978125 15470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "e5f3c854f5a1404ebea932673f1ed717"
format_stamp: "Formatted at 2026-08-10 07:23:56 on dist-test-slave-x5fp"
server_key: "0c8bdd3b80185df472839dc72939fb59"
server_key_iv: "7bc3fa9b7376524b9becf429c9d1b82a"
server_key_version: "encryptionkey@0"
I20260810 07:23:56.981134 15470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:56.982372 15489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:56.982592 15470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:23:56.982708 15470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "e5f3c854f5a1404ebea932673f1ed717"
format_stamp: "Formatted at 2026-08-10 07:23:56 on dist-test-slave-x5fp"
server_key: "0c8bdd3b80185df472839dc72939fb59"
server_key_iv: "7bc3fa9b7376524b9becf429c9d1b82a"
server_key_version: "encryptionkey@0"
I20260810 07:23:56.982811 15470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:57.002830 15470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:57.005298 15470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:57.005491 15470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:57.011256 15470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33157
I20260810 07:23:57.011291 15579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33157 every 8 connection(s)
I20260810 07:23:57.011709 15470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:57.012650 15581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:57.015090 15581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717: Bootstrap starting.
I20260810 07:23:57.015640 15581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:57.015977 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15470
I20260810 07:23:57.015974 15581 log.cc:824] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:57.016098 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:57.016311 14992 external_mini_cluster.cc:1428] Setting key 26a1f711aa3277de58a9b7ed0313d173
I20260810 07:23:57.017040 15581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717: No bootstrap required, opened a new log
I20260810 07:23:57.018764 15581 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f3c854f5a1404ebea932673f1ed717" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33157 } }
I20260810 07:23:57.018918 15581 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:57.018941 15581 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f3c854f5a1404ebea932673f1ed717, State: Initialized, Role: FOLLOWER
I20260810 07:23:57.019080 15581 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f3c854f5a1404ebea932673f1ed717" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33157 } }
I20260810 07:23:57.019181 15581 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:57.019225 15581 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:57.019299 15581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:57.020118 15581 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f3c854f5a1404ebea932673f1ed717" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33157 } }
I20260810 07:23:57.020236 15581 leader_election.cc:302] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5f3c854f5a1404ebea932673f1ed717; no voters:
I20260810 07:23:57.020421 15581 leader_election.cc:288] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:57.020481 15591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:57.020686 15591 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [term 1 LEADER]: Becoming Leader. State: Replica: e5f3c854f5a1404ebea932673f1ed717, State: Running, Role: LEADER
I20260810 07:23:57.020704 15581 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:57.020824 15591 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f3c854f5a1404ebea932673f1ed717" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33157 } }
I20260810 07:23:57.021214 15591 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5f3c854f5a1404ebea932673f1ed717. Latest consensus state: current_term: 1 leader_uuid: "e5f3c854f5a1404ebea932673f1ed717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f3c854f5a1404ebea932673f1ed717" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33157 } } }
I20260810 07:23:57.021311 15591 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:57.021595 15602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:57.021222 15592 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5f3c854f5a1404ebea932673f1ed717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f3c854f5a1404ebea932673f1ed717" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33157 } } }
I20260810 07:23:57.022302 15592 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:57.022835 15602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:57.025023 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:57.016943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:39071 (local address 127.0.0.1:33157)
0810 07:23:57.017218 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:57.017227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:57.017286 (+ 59us) server_negotiation.cc:407] Connection header received
0810 07:23:57.018808 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:57.018818 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:57.018878 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:57.019060 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:57.019363 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:57.021213 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:57.022167 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:57.022800 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:57.022908 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:57.024134 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:57.024138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:57.024138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:57.024375 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:57.024429 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:57.024444 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:57.024556 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:57.024788 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 07:23:57.024827 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":121,"threads_started":1}
I20260810 07:23:57.025530 15602 catalog_manager.cc:1347] Generated new cluster ID: a2953c7680fb44979506e0edeafc1ddb
I20260810 07:23:57.025584 15602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:57.026113 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:57.031766 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:57.027869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:53397 (local address 127.0.0.1:33157)
0810 07:23:57.027926 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:57.027928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:57.028118 (+ 190us) server_negotiation.cc:407] Connection header received
0810 07:23:57.028185 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:57.028186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:57.028197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:57.028246 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:57.028410 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:57.029222 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:57.030330 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:57.030654 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:57.030717 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:57.031488 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:57.031489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:57.031490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:57.031494 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:57.031510 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:57.031515 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:57.031563 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:57.031671 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 07:23:57.031683 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:23:57.040206 15602 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:57.040844 15602 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:57.044982 15602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5f3c854f5a1404ebea932673f1ed717: Generated new TSK 0
I20260810 07:23:57.045195 15602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:57.049736 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15470
W20260810 07:23:57.061900 15449 connection.cc:535] client connection to 127.0.0.1:33157 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33157): Connection reset by peer (error 104)
2026-08-10T07:23:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:57 dist-test-slave-x5fp krb5kdc[15626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:57 dist-test-slave-x5fp krb5kdc[15626](info): set up 2 sockets
Aug 10 07:23:57 dist-test-slave-x5fp krb5kdc[15626](info): commencing operation
krb5kdc: starting...
W20260810 07:23:58.340799 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:23:58 dist-test-slave-x5fp krb5kdc[15626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:23:58.485989 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:58.571040 15642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:58.571308 15642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:58.571350 15642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:58.572611 15642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:58.572696 15642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:58.572721 15642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:58.572798 15642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:58.572830 15642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:58.574396 15642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:23:58.574709 15642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:58.574963 15642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:58.579999 15654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:58.580018 15657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:58.579988 15655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:58.580803 15642 server_base.cc:1034] running on GCE node
I20260810 07:23:58.581077 15642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:58.581338 15642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:58.582532 15642 hybrid_clock.cc:648] HybridClock initialized: now 1786346638582513 us; error 34 us; skew 500 ppm
Aug 10 07:23:58 dist-test-slave-x5fp krb5kdc[15626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:58.591156 15642 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:23:58.591774 15642 webserver.cc:460] Webserver started at http://127.0.0.1:40621/ using document root <none> and password file <none>
I20260810 07:23:58.592065 15642 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:58.592146 15642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:58.592326 15642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:58.593729 15642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "7c1c745fdcd04375ac05e5dda62e9c8d"
format_stamp: "Formatted at 2026-08-10 07:23:58 on dist-test-slave-x5fp"
server_key: "c50dae4c8ca97a2dfaaa8992cb1ab98e"
server_key_iv: "1db7e087aaf5e9db6782aa71f2881e09"
server_key_version: "encryptionkey@0"
I20260810 07:23:58.594165 15642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "7c1c745fdcd04375ac05e5dda62e9c8d"
format_stamp: "Formatted at 2026-08-10 07:23:58 on dist-test-slave-x5fp"
server_key: "c50dae4c8ca97a2dfaaa8992cb1ab98e"
server_key_iv: "1db7e087aaf5e9db6782aa71f2881e09"
server_key_version: "encryptionkey@0"
I20260810 07:23:58.595913 15642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:23:58.596957 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:58.597167 15642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:23:58.597319 15642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "7c1c745fdcd04375ac05e5dda62e9c8d"
format_stamp: "Formatted at 2026-08-10 07:23:58 on dist-test-slave-x5fp"
server_key: "c50dae4c8ca97a2dfaaa8992cb1ab98e"
server_key_iv: "1db7e087aaf5e9db6782aa71f2881e09"
server_key_version: "encryptionkey@0"
I20260810 07:23:58.597409 15642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:58.601325 15642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:58.604079 15642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:58.604282 15642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:58.608876 15642 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39865
I20260810 07:23:58.608917 15750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39865 every 8 connection(s)
I20260810 07:23:58.609316 15642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:58.610491 15751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:58.612310 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15642
I20260810 07:23:58.612455 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:23:58.612771 14992 external_mini_cluster.cc:1428] Setting key ef278466a6835007d080a3b8e13093a4
I20260810 07:23:58.613771 15751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d: Bootstrap starting.
I20260810 07:23:58.614562 15751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:58.614964 15751 log.cc:824] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:58.615731 15751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d: No bootstrap required, opened a new log
I20260810 07:23:58.616118 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:58.613416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:42529 (local address 127.0.0.1:39865)
0810 07:23:58.613649 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:58.613660 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:58.613698 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:23:58.614341 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:58.614351 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:58.614388 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:58.614566 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:58.615554 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:58.615558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:58.615559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:58.615564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:58.615618 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:58.615630 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:58.615814 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:58.615859 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 07:23:58.615914 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":101,"threads_started":1}
I20260810 07:23:58.617442 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:23:58.618746 15751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39865 } }
I20260810 07:23:58.619014 15751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:58.619076 15751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1c745fdcd04375ac05e5dda62e9c8d, State: Initialized, Role: FOLLOWER
I20260810 07:23:58.619238 15751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39865 } }
I20260810 07:23:58.619457 15751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:58.619551 15751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:58.619617 15751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:58.620589 15751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39865 } }
I20260810 07:23:58.620743 15751 leader_election.cc:302] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c1c745fdcd04375ac05e5dda62e9c8d; no voters:
I20260810 07:23:58.620962 15767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:58.620939 15751 leader_election.cc:288] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:58.621130 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:58.619382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:35439 (local address 127.0.0.1:39865)
0810 07:23:58.619427 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:58.619429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:58.619561 (+ 132us) server_negotiation.cc:407] Connection header received
0810 07:23:58.619662 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:58.619663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:58.619670 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:58.619762 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:58.620518 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:58.620520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:58.620520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:23:58.620525 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:58.620545 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:23:58.620550 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:58.620627 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:58.620822 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 07:23:58.620838 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:23:58.621227 15767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [term 1 LEADER]: Becoming Leader. State: Replica: 7c1c745fdcd04375ac05e5dda62e9c8d, State: Running, Role: LEADER
I20260810 07:23:58.621393 15751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:58.621321 15767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39865 } }
I20260810 07:23:58.621899 15767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c1c745fdcd04375ac05e5dda62e9c8d. Latest consensus state: current_term: 1 leader_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39865 } } }
I20260810 07:23:58.622005 15767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:58.622164 15767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c745fdcd04375ac05e5dda62e9c8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39865 } } }
I20260810 07:23:58.622231 15767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:58.623365 15785 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:58.623451 15785 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:58.623515 15777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:58.623674 15777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:58.625367 15777 catalog_manager.cc:1347] Generated new cluster ID: d60bec03d5ba4555b5196034a904817e
I20260810 07:23:58.625424 15777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:58.644373 15777 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:58.644613 15777 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:58.655272 15777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c1c745fdcd04375ac05e5dda62e9c8d: Generated new TSK 0
I20260810 07:23:58.655547 15777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:23:58.660324 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15642
2026-08-10T07:23:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:58 dist-test-slave-x5fp krb5kdc[15793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:58 dist-test-slave-x5fp krb5kdc[15793](info): set up 2 sockets
Aug 10 07:23:58 dist-test-slave-x5fp krb5kdc[15793](info): commencing operation
krb5kdc: starting...
W20260810 07:23:59.927371 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:00 dist-test-slave-x5fp krb5kdc[15793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:00.071740 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:00.157903 15811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:00.158128 15811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:00.158150 15811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:00.159257 15811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:00.159296 15811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:00.159312 15811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:00.159324 15811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:00.159337 15811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:00.160794 15811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:00.161005 15811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:00.161188 15811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:00.166515 15819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:00.166520 15817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:00.166647 15822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:00.166931 15811 server_base.cc:1034] running on GCE node
I20260810 07:24:00.167258 15811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:00.167583 15811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:00.168789 15811 hybrid_clock.cc:648] HybridClock initialized: now 1786346640168770 us; error 52 us; skew 500 ppm
Aug 10 07:24:00 dist-test-slave-x5fp krb5kdc[15793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:00.171377 15811 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:00.171869 15811 webserver.cc:460] Webserver started at http://127.0.0.1:34237/ using document root <none> and password file <none>
I20260810 07:24:00.172102 15811 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:00.172173 15811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:00.172310 15811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:00.173483 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "fe3a90e0d28a4e85a854df2bc3a109fe"
format_stamp: "Formatted at 2026-08-10 07:24:00 on dist-test-slave-x5fp"
server_key: "28a61e0935407f25c3f151ef5cc96cc3"
server_key_iv: "b730a2beb6a6c2ea48f9e8ceab3f9405"
server_key_version: "encryptionkey@0"
I20260810 07:24:00.173858 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "fe3a90e0d28a4e85a854df2bc3a109fe"
format_stamp: "Formatted at 2026-08-10 07:24:00 on dist-test-slave-x5fp"
server_key: "28a61e0935407f25c3f151ef5cc96cc3"
server_key_iv: "b730a2beb6a6c2ea48f9e8ceab3f9405"
server_key_version: "encryptionkey@0"
I20260810 07:24:00.175243 15811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:24:00.176054 15829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:00.176244 15811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:00.176370 15811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "fe3a90e0d28a4e85a854df2bc3a109fe"
format_stamp: "Formatted at 2026-08-10 07:24:00 on dist-test-slave-x5fp"
server_key: "28a61e0935407f25c3f151ef5cc96cc3"
server_key_iv: "b730a2beb6a6c2ea48f9e8ceab3f9405"
server_key_version: "encryptionkey@0"
I20260810 07:24:00.176450 15811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:00.185137 15811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:00.187068 15811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:00.187237 15811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:00.191452 15811 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44037
I20260810 07:24:00.191486 15913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44037 every 8 connection(s)
I20260810 07:24:00.191838 15811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:00.192776 15914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:00.195098 15914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe: Bootstrap starting.
I20260810 07:24:00.195636 15914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:00.195879 15914 log.cc:824] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:00.196597 15914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe: No bootstrap required, opened a new log
I20260810 07:24:00.197992 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15811
I20260810 07:24:00.198125 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:00.198334 14992 external_mini_cluster.cc:1428] Setting key 028c34231f6a550fe9db7bc576e346e9
I20260810 07:24:00.198701 15914 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260810 07:24:00.198897 15914 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:00.198954 15914 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3a90e0d28a4e85a854df2bc3a109fe, State: Initialized, Role: FOLLOWER
I20260810 07:24:00.199110 15914 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260810 07:24:00.199208 15914 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:00.199244 15914 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:00.199294 15914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:00.200387 15914 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260810 07:24:00.200500 15914 leader_election.cc:302] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe3a90e0d28a4e85a854df2bc3a109fe; no voters:
I20260810 07:24:00.200660 15914 leader_election.cc:288] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:00.200804 15921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:00.200970 15914 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:00.201040 15921 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [term 1 LEADER]: Becoming Leader. State: Replica: fe3a90e0d28a4e85a854df2bc3a109fe, State: Running, Role: LEADER
I20260810 07:24:00.201211 15921 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } }
I20260810 07:24:00.201678 15921 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe3a90e0d28a4e85a854df2bc3a109fe. Latest consensus state: current_term: 1 leader_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } } }
I20260810 07:24:00.201687 15924 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3a90e0d28a4e85a854df2bc3a109fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44037 } } }
I20260810 07:24:00.201906 15921 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:00.201915 15924 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe [sys.catalog]: This master's current role is: LEADER
W20260810 07:24:00.202781 15943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:00.203078 15943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:00.203154 15946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:00.203323 15946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:00.205090 15946 catalog_manager.cc:1347] Generated new cluster ID: 10b9cf5f0bbf4bcba8c4764e23dc26fd
I20260810 07:24:00.205156 15946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:24:00 dist-test-slave-x5fp krb5kdc[15793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:24:00.210520 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:00.198902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:55277 (local address 127.0.0.1:44037)
0810 07:24:00.199117 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:00.199127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:00.199195 (+ 68us) server_negotiation.cc:407] Connection header received
0810 07:24:00.199428 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:00.199436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:00.199497 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:00.199654 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:00.200380 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:00.201867 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:00.203094 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:00.203755 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:00.203886 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:00.206666 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:00.206670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:00.206670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:00.206683 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:00.208409 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:00.208859 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:00.208867 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:00.208867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:00.208972 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:00.209221 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:00.209222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:00.209223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:00.209466 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:00.210203 (+ 737us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:00.210277 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 07:24:00.210362 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260810 07:24:00.211616 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:00.217700 15946 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:00.218469 15946 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:00.227103 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:00.217141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:56091 (local address 127.0.0.1:44037)
0810 07:24:00.217205 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:00.217207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:00.217349 (+ 142us) server_negotiation.cc:407] Connection header received
0810 07:24:00.217444 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:00.217444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:00.217453 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:00.217504 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:00.221294 (+ 3790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:00.221790 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:00.222568 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:00.222782 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:00.222857 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:00.224270 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:00.224272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:00.224272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:00.224280 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:00.225852 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:00.226251 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:00.226252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:00.226252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:00.226314 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:00.226554 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:00.226555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:00.226555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:00.226668 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:00.226782 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:00.226910 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 07:24:00.226990 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:24:00.239161 15946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe3a90e0d28a4e85a854df2bc3a109fe: Generated new TSK 0
I20260810 07:24:00.239408 15946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:00.247843 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15811
W20260810 07:24:00.267002 15791 connection.cc:535] client connection to 127.0.0.1:44037 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44037): Connection reset by peer (error 104)
2026-08-10T07:24:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1595 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:00 dist-test-slave-x5fp krb5kdc[15969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:00 dist-test-slave-x5fp krb5kdc[15969](info): set up 2 sockets
Aug 10 07:24:00 dist-test-slave-x5fp krb5kdc[15969](info): commencing operation
krb5kdc: starting...
W20260810 07:24:01.544152 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:01 dist-test-slave-x5fp krb5kdc[15969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:01.675827 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46345
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:01.762431 15991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:01.762652 15991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:01.762676 15991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:01.763931 15991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:01.763980 15991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:01.763995 15991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:01.764008 15991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:01.764020 15991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:01.765542 15991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46345
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:01.765820 15991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:01.766052 15991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:01.771324 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:01.771377 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:01.771476 15991 server_base.cc:1034] running on GCE node
W20260810 07:24:01.771656 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:01.772013 15991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:01.772378 15991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:01.773550 15991 hybrid_clock.cc:648] HybridClock initialized: now 1786346641773524 us; error 37 us; skew 500 ppm
Aug 10 07:24:01 dist-test-slave-x5fp krb5kdc[15969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:01.776311 15991 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:01.776902 15991 webserver.cc:460] Webserver started at http://127.0.0.1:46401/ using document root <none> and password file <none>
I20260810 07:24:01.777138 15991 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:01.777186 15991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:01.777323 15991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:01.778487 15991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d"
format_stamp: "Formatted at 2026-08-10 07:24:01 on dist-test-slave-x5fp"
server_key: "9fdb5596cdbbfb12115526b703c39005"
server_key_iv: "e1c98dfbd6d90b8f842cb252d9b1e280"
server_key_version: "encryptionkey@0"
I20260810 07:24:01.778882 15991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d"
format_stamp: "Formatted at 2026-08-10 07:24:01 on dist-test-slave-x5fp"
server_key: "9fdb5596cdbbfb12115526b703c39005"
server_key_iv: "e1c98dfbd6d90b8f842cb252d9b1e280"
server_key_version: "encryptionkey@0"
I20260810 07:24:01.780311 15991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:24:01.781241 16011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:01.781423 15991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:24:01.781538 15991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d"
format_stamp: "Formatted at 2026-08-10 07:24:01 on dist-test-slave-x5fp"
server_key: "9fdb5596cdbbfb12115526b703c39005"
server_key_iv: "e1c98dfbd6d90b8f842cb252d9b1e280"
server_key_version: "encryptionkey@0"
I20260810 07:24:01.781615 15991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:01.802171 15991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:01.804991 15991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:01.805186 15991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:01.809777 15991 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46345
I20260810 07:24:01.809825 16085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46345 every 8 connection(s)
I20260810 07:24:01.810243 15991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:01.811247 16086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:01.811827 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 15991
I20260810 07:24:01.811951 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:01.812196 14992 external_mini_cluster.cc:1428] Setting key b5f17fbce791d1383b7f0c9d29e9ba2f
I20260810 07:24:01.814615 16086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d: Bootstrap starting.
I20260810 07:24:01.815301 16086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:01.815663 16086 log.cc:824] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:01.816496 16086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d: No bootstrap required, opened a new log
I20260810 07:24:01.818720 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:01.812920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:33819 (local address 127.0.0.1:46345)
0810 07:24:01.813132 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:01.813140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:01.813169 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:24:01.813651 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:01.813658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:01.813715 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:01.813852 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:01.814144 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:01.815213 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:01.816357 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:01.817011 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:01.817110 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:01.817946 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:01.817950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:01.817950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:01.818134 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:01.818168 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:01.818179 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:01.818260 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:01.818400 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 07:24:01.818463 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260810 07:24:01.819041 16086 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46345 } }
I20260810 07:24:01.819186 16086 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:01.819209 16086 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0f28d39ac54aa08b0e58a06ad1c28d, State: Initialized, Role: FOLLOWER
I20260810 07:24:01.819366 16086 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46345 } }
I20260810 07:24:01.819433 16086 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:01.819455 16086 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:01.819484 16086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:01.819991 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:01.820241 16086 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46345 } }
I20260810 07:24:01.820353 16086 leader_election.cc:302] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc0f28d39ac54aa08b0e58a06ad1c28d; no voters:
I20260810 07:24:01.820529 16086 leader_election.cc:288] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:01.820591 16095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:01.820822 16086 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:01.820832 16095 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [term 1 LEADER]: Becoming Leader. State: Replica: cc0f28d39ac54aa08b0e58a06ad1c28d, State: Running, Role: LEADER
I20260810 07:24:01.821069 16095 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46345 } }
I20260810 07:24:01.821528 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc0f28d39ac54aa08b0e58a06ad1c28d. Latest consensus state: current_term: 1 leader_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46345 } } }
I20260810 07:24:01.821632 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:01.821537 16097 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0f28d39ac54aa08b0e58a06ad1c28d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46345 } } }
I20260810 07:24:01.822039 16097 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:01.822263 16118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:01.823073 16118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:01.824761 16118 catalog_manager.cc:1347] Generated new cluster ID: 3318bd88c58c454284c0ecf6d949cf3d
I20260810 07:24:01.824875 16118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:01.825754 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:01.821926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:41647 (local address 127.0.0.1:46345)
0810 07:24:01.821961 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:01.821963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:01.821974 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:24:01.822061 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:01.822063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:01.822075 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:01.822108 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:01.822365 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:01.823125 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:01.824206 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:01.824501 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:01.824606 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:01.825451 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:01.825452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:01.825453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:01.825457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:01.825473 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:01.825478 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:01.825549 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:01.825662 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 07:24:01.825674 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 07:24:01.837011 16118 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:01.837579 16118 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:01.841085 16118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc0f28d39ac54aa08b0e58a06ad1c28d: Generated new TSK 0
I20260810 07:24:01.841287 16118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:01.846738 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 15991
W20260810 07:24:01.864319 15962 connection.cc:535] client connection to 127.0.0.1:46345 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46345): Connection reset by peer (error 104)
2026-08-10T07:24:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:01 dist-test-slave-x5fp krb5kdc[16137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:01 dist-test-slave-x5fp krb5kdc[16137](info): set up 2 sockets
Aug 10 07:24:01 dist-test-slave-x5fp krb5kdc[16137](info): commencing operation
krb5kdc: starting...
W20260810 07:24:03.150625 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:03 dist-test-slave-x5fp krb5kdc[16137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:03.284483 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:03.370026 16168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:03.370272 16168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:03.370299 16168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:03.371497 16168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:03.371541 16168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:03.371556 16168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:03.371569 16168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:03.371582 16168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:03.373798 16168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:03.374082 16168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:03.374305 16168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:03.380048 16178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:03.380041 16173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:03.380059 16175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:03.380467 16168 server_base.cc:1034] running on GCE node
I20260810 07:24:03.380808 16168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:03.381155 16168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:03.382347 16168 hybrid_clock.cc:648] HybridClock initialized: now 1786346643382324 us; error 42 us; skew 500 ppm
Aug 10 07:24:03 dist-test-slave-x5fp krb5kdc[16137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:03.385097 16168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:03.385658 16168 webserver.cc:460] Webserver started at http://127.0.0.1:43403/ using document root <none> and password file <none>
I20260810 07:24:03.385938 16168 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:03.386009 16168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:03.386168 16168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:03.387341 16168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "196ebb8771f0461486e9b50529fab8ac"
format_stamp: "Formatted at 2026-08-10 07:24:03 on dist-test-slave-x5fp"
server_key: "0c43cb1f6739a58f7924ed50cbbe9329"
server_key_iv: "c7113205f7a003f9b24d832a01694430"
server_key_version: "encryptionkey@0"
I20260810 07:24:03.387738 16168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "196ebb8771f0461486e9b50529fab8ac"
format_stamp: "Formatted at 2026-08-10 07:24:03 on dist-test-slave-x5fp"
server_key: "0c43cb1f6739a58f7924ed50cbbe9329"
server_key_iv: "c7113205f7a003f9b24d832a01694430"
server_key_version: "encryptionkey@0"
I20260810 07:24:03.389267 16168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:24:03.390237 16189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:03.390486 16168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:03.390583 16168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "196ebb8771f0461486e9b50529fab8ac"
format_stamp: "Formatted at 2026-08-10 07:24:03 on dist-test-slave-x5fp"
server_key: "0c43cb1f6739a58f7924ed50cbbe9329"
server_key_iv: "c7113205f7a003f9b24d832a01694430"
server_key_version: "encryptionkey@0"
I20260810 07:24:03.390676 16168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:03.394320 16168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:03.396389 16168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:03.396554 16168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:03.400628 16168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45869
I20260810 07:24:03.400686 16273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45869 every 8 connection(s)
I20260810 07:24:03.401069 16168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:03.401986 16274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:03.404333 16274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac: Bootstrap starting.
I20260810 07:24:03.405001 16274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:03.405290 16274 log.cc:824] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:03.406023 16274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac: No bootstrap required, opened a new log
I20260810 07:24:03.407661 16274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ebb8771f0461486e9b50529fab8ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260810 07:24:03.407835 16274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:03.407881 16274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196ebb8771f0461486e9b50529fab8ac, State: Initialized, Role: FOLLOWER
I20260810 07:24:03.408074 16274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ebb8771f0461486e9b50529fab8ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260810 07:24:03.408176 16274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:03.408224 16274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:03.408277 16274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:03.409221 16274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ebb8771f0461486e9b50529fab8ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260810 07:24:03.409344 16274 leader_election.cc:302] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 196ebb8771f0461486e9b50529fab8ac; no voters:
I20260810 07:24:03.409526 16274 leader_election.cc:288] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:03.409612 16280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:03.409798 16274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:03.409884 16280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [term 1 LEADER]: Becoming Leader. State: Replica: 196ebb8771f0461486e9b50529fab8ac, State: Running, Role: LEADER
I20260810 07:24:03.409999 16280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ebb8771f0461486e9b50529fab8ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260810 07:24:03.410571 16283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "196ebb8771f0461486e9b50529fab8ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ebb8771f0461486e9b50529fab8ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } } }
I20260810 07:24:03.410573 16284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 196ebb8771f0461486e9b50529fab8ac. Latest consensus state: current_term: 1 leader_uuid: "196ebb8771f0461486e9b50529fab8ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ebb8771f0461486e9b50529fab8ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } } }
I20260810 07:24:03.410706 16283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:03.410737 16284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:03.410892 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16168
I20260810 07:24:03.410976 16295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:03.411032 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:03.411262 14992 external_mini_cluster.cc:1428] Setting key 2669e1354d138fa5530ec77ae194b903
I20260810 07:24:03.411520 16295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:03.413551 16295 catalog_manager.cc:1347] Generated new cluster ID: abfbd7b18e3f46b7a5e5ec1917785228
I20260810 07:24:03.413611 16295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:03.414131 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:03.412130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:55725 (local address 127.0.0.1:45869)
0810 07:24:03.412352 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:03.412363 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:03.412400 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:24:03.412544 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:03.412552 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:03.412585 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:03.412796 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:03.413510 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:03.413514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:03.413514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:03.413520 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:03.413572 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:03.413585 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:03.413878 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:03.413919 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:24:03.413971 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260810 07:24:03.415180 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:03.418051 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:03.416713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:50731 (local address 127.0.0.1:45869)
0810 07:24:03.416776 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:03.416779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:03.416943 (+ 164us) server_negotiation.cc:407] Connection header received
0810 07:24:03.417014 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:03.417015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:03.417022 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:03.417084 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:03.417783 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:03.417784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:03.417785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:03.417789 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:03.417804 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:03.417809 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:03.417866 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:03.417939 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:24:03.417956 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:24:03.418370 16295 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:03.418566 16295 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:03.435377 16295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 196ebb8771f0461486e9b50529fab8ac: Generated new TSK 0
I20260810 07:24:03.435614 16295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:03.438410 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16168
2026-08-10T07:24:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:03 dist-test-slave-x5fp krb5kdc[16341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:03 dist-test-slave-x5fp krb5kdc[16341](info): set up 2 sockets
Aug 10 07:24:03 dist-test-slave-x5fp krb5kdc[16341](info): commencing operation
krb5kdc: starting...
W20260810 07:24:04.719933 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:04 dist-test-slave-x5fp krb5kdc[16341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:04.855240 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42323
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:42323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:04.942977 16360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:04.943274 16360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:04.943320 16360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:04.944515 16360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:04.944598 16360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:04.944626 16360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:04.944651 16360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:04.944682 16360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:04.946300 16360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:42323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:42323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:04.946620 16360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:04.946887 16360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:04.952090 16375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.952142 16368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.952126 16369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:04.952270 16360 server_base.cc:1034] running on GCE node
I20260810 07:24:04.952705 16360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:04.953063 16360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:04.954238 16360 hybrid_clock.cc:648] HybridClock initialized: now 1786346644954220 us; error 29 us; skew 500 ppm
Aug 10 07:24:04 dist-test-slave-x5fp krb5kdc[16341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:04.961417 16360 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:04.961988 16360 webserver.cc:460] Webserver started at http://127.14.164.62:37861/ using document root <none> and password file <none>
I20260810 07:24:04.962267 16360 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:04.962349 16360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:04.962523 16360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:04.963750 16360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "00351172c02b4514945f6f0d09ee487a"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-x5fp"
server_key: "67b81f38617880f0358b655653fbaed2"
server_key_iv: "dff74e24a15164723f3b7579048e82bd"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.964172 16360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "00351172c02b4514945f6f0d09ee487a"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-x5fp"
server_key: "67b81f38617880f0358b655653fbaed2"
server_key_iv: "dff74e24a15164723f3b7579048e82bd"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.965817 16360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 07:24:04.966807 16385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.967079 16360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:24:04.967211 16360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "00351172c02b4514945f6f0d09ee487a"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-x5fp"
server_key: "67b81f38617880f0358b655653fbaed2"
server_key_iv: "dff74e24a15164723f3b7579048e82bd"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.967307 16360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:04.977771 16360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:04.980317 16360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:04.980535 16360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:04.985205 16360 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:42323
I20260810 07:24:04.985280 16477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:42323 every 8 connection(s)
I20260810 07:24:04.985818 16360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:04.987118 16478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:04.989876 16478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a: Bootstrap starting.
I20260810 07:24:04.990568 16478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:04.990914 16478 log.cc:824] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:04.991678 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16360
I20260810 07:24:04.991880 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:04.991833 16478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a: No bootstrap required, opened a new log
I20260810 07:24:04.992110 14992 external_mini_cluster.cc:1428] Setting key 4d9235124b52aada1fa14f7c79d184f8
I20260810 07:24:04.994904 16478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00351172c02b4514945f6f0d09ee487a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42323 } }
I20260810 07:24:04.995078 16478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:04.995105 16478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00351172c02b4514945f6f0d09ee487a, State: Initialized, Role: FOLLOWER
I20260810 07:24:04.995254 16478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00351172c02b4514945f6f0d09ee487a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42323 } }
I20260810 07:24:04.995338 16478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:04.995363 16478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:04.995445 16478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:04.996143 16478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00351172c02b4514945f6f0d09ee487a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42323 } }
I20260810 07:24:04.996245 16478 leader_election.cc:302] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00351172c02b4514945f6f0d09ee487a; no voters:
I20260810 07:24:04.996531 16478 leader_election.cc:288] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:04.996558 16485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:04.996796 16485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [term 1 LEADER]: Becoming Leader. State: Replica: 00351172c02b4514945f6f0d09ee487a, State: Running, Role: LEADER
I20260810 07:24:04.996975 16478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:04.996973 16485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00351172c02b4514945f6f0d09ee487a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42323 } }
I20260810 07:24:04.997481 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00351172c02b4514945f6f0d09ee487a. Latest consensus state: current_term: 1 leader_uuid: "00351172c02b4514945f6f0d09ee487a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00351172c02b4514945f6f0d09ee487a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42323 } } }
I20260810 07:24:04.997605 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:04.997826 16487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00351172c02b4514945f6f0d09ee487a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00351172c02b4514945f6f0d09ee487a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 42323 } } }
I20260810 07:24:04.997936 16487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:04.998456 16505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:04.998996 16505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:24:05 dist-test-slave-x5fp krb5kdc[16341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:05.002187 16505 catalog_manager.cc:1347] Generated new cluster ID: e54499cb8b694f09b0732377774bf9a4
I20260810 07:24:05.002239 16505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:05.005771 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.992802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37043 (local address 127.14.164.62:42323)
0810 07:24:04.993041 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.993051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.993100 (+ 49us) server_negotiation.cc:407] Connection header received
0810 07:24:04.993634 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.993642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.993696 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.993823 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:04.994861 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.996485 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.997821 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.998519 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.998638 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.001773 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.001777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.001778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:05.001793 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.003952 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:05.004443 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:05.004447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:05.004447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:05.004541 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:05.004847 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:05.004847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:05.004848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:05.005113 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.005296 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.005452 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.005567 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":158,"threads_started":1}
I20260810 07:24:05.006848 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:05.012681 16505 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:05.013517 16505 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:05.015735 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.008503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43541 (local address 127.14.164.62:42323)
0810 07:24:05.008545 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.008547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.008557 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:24:05.008624 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.008625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.008635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.008697 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:05.009302 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.009871 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.010682 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.010938 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.011037 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.012635 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.012636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.012636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:05.012644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.014315 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:05.014726 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:05.014727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:05.014727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:05.014800 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:05.015052 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:05.015053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:05.015054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:05.015181 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.015309 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.015463 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.015536 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:24:05.020431 16505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00351172c02b4514945f6f0d09ee487a: Generated new TSK 0
I20260810 07:24:05.020643 16505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:05.028528 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16360
W20260810 07:24:05.044878 16333 connection.cc:535] client connection to 127.14.164.62:42323 recv error: Network error: failed to read from TLS socket (remote: 127.14.164.62:42323): Connection reset by peer (error 104)
2026-08-10T07:24:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:05 dist-test-slave-x5fp krb5kdc[16528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:05 dist-test-slave-x5fp krb5kdc[16528](info): set up 2 sockets
Aug 10 07:24:05 dist-test-slave-x5fp krb5kdc[16528](info): commencing operation
krb5kdc: starting...
W20260810 07:24:06.329217 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:06 dist-test-slave-x5fp krb5kdc[16528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:06.465370 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40603
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:40603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:06.552188 16550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:06.552425 16550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:06.552470 16550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:06.553712 16550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:06.553762 16550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:06.553777 16550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:06.553799 16550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:06.553812 16550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:06.555379 16550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:40603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:06.555680 16550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:06.555944 16550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:06.561525 16558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:06.561542 16556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:06.561538 16555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:06.561615 16550 server_base.cc:1034] running on GCE node
I20260810 07:24:06.562167 16550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:06.562528 16550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:06.563745 16550 hybrid_clock.cc:648] HybridClock initialized: now 1786346646563722 us; error 68 us; skew 500 ppm
Aug 10 07:24:06 dist-test-slave-x5fp krb5kdc[16528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346646, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:06.566823 16550 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:06.567324 16550 webserver.cc:460] Webserver started at http://127.14.164.62:36429/ using document root <none> and password file <none>
I20260810 07:24:06.567588 16550 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:06.567664 16550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:06.567832 16550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:06.569082 16550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "7a39f895b7244073a3fc8b18a19c2d7a"
format_stamp: "Formatted at 2026-08-10 07:24:06 on dist-test-slave-x5fp"
server_key: "85f7f937c281846beab1f9f571778338"
server_key_iv: "0214a05ec8b0053253f2e86d31b21b9c"
server_key_version: "encryptionkey@0"
I20260810 07:24:06.569476 16550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "7a39f895b7244073a3fc8b18a19c2d7a"
format_stamp: "Formatted at 2026-08-10 07:24:06 on dist-test-slave-x5fp"
server_key: "85f7f937c281846beab1f9f571778338"
server_key_iv: "0214a05ec8b0053253f2e86d31b21b9c"
server_key_version: "encryptionkey@0"
I20260810 07:24:06.570962 16550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:24:06.571821 16567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:06.572117 16550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:06.572304 16550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "7a39f895b7244073a3fc8b18a19c2d7a"
format_stamp: "Formatted at 2026-08-10 07:24:06 on dist-test-slave-x5fp"
server_key: "85f7f937c281846beab1f9f571778338"
server_key_iv: "0214a05ec8b0053253f2e86d31b21b9c"
server_key_version: "encryptionkey@0"
I20260810 07:24:06.572402 16550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:06.591956 16550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:06.594674 16550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:06.594907 16550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:06.599630 16550 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:40603
I20260810 07:24:06.599869 16640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:40603 every 8 connection(s)
I20260810 07:24:06.600155 16550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:06.601455 16641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:06.602222 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16550
I20260810 07:24:06.602418 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:06.602666 14992 external_mini_cluster.cc:1428] Setting key afddd31de8abae41c09bd3df5b5da912
I20260810 07:24:06.605083 16641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a: Bootstrap starting.
I20260810 07:24:06.606176 16641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:06.606549 16641 log.cc:824] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:06.607686 16641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a: No bootstrap required, opened a new log
I20260810 07:24:06.609871 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.603359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47685 (local address 127.14.164.62:40603)
0810 07:24:06.603595 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:06.603604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:06.603636 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:24:06.604120 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:06.604128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:06.604182 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:06.604320 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:06.604932 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.606007 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.607183 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.607674 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.607775 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.608799 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:06.608805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:06.608805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:06.609018 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:06.609057 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:06.609070 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:06.609147 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:06.609319 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 07:24:06.609470 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260810 07:24:06.610025 16641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40603 } }
I20260810 07:24:06.610270 16641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:06.610332 16641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a39f895b7244073a3fc8b18a19c2d7a, State: Initialized, Role: FOLLOWER
I20260810 07:24:06.610541 16641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40603 } }
I20260810 07:24:06.610654 16641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:06.610707 16641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:06.610766 16641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:06.611068 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:06.611552 16641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40603 } }
I20260810 07:24:06.611701 16641 leader_election.cc:302] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a39f895b7244073a3fc8b18a19c2d7a; no voters:
I20260810 07:24:06.612179 16641 leader_election.cc:288] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:06.612411 16651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:06.612625 16641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:06.612914 16651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [term 1 LEADER]: Becoming Leader. State: Replica: 7a39f895b7244073a3fc8b18a19c2d7a, State: Running, Role: LEADER
I20260810 07:24:06.613045 16651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40603 } }
I20260810 07:24:06.613453 16651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a39f895b7244073a3fc8b18a19c2d7a. Latest consensus state: current_term: 1 leader_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40603 } } }
I20260810 07:24:06.613476 16653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a39f895b7244073a3fc8b18a19c2d7a" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40603 } } }
I20260810 07:24:06.613672 16651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:06.613724 16653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:06.613956 16668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:06.614614 16668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:06.616788 16668 catalog_manager.cc:1347] Generated new cluster ID: 6be42fa351db4c3184f18037430e913f
I20260810 07:24:06.616856 16668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:06.618687 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.612674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57201 (local address 127.14.164.62:40603)
0810 07:24:06.612733 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:06.612735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:06.614689 (+ 1954us) server_negotiation.cc:407] Connection header received
0810 07:24:06.614829 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:06.614830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:06.614844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:06.614908 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:06.615173 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.615935 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.616855 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.617190 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.617269 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.618338 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:06.618339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:06.618339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:06.618361 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:06.618379 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:06.618383 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:06.618465 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:06.618585 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 07:24:06.618598 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:24:06.628814 16668 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:06.629693 16668 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:06.634251 16668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a39f895b7244073a3fc8b18a19c2d7a: Generated new TSK 0
I20260810 07:24:06.634493 16668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:06.643951 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16550
W20260810 07:24:06.656809 16525 connection.cc:535] client connection to 127.14.164.62:40603 recv error: Network error: failed to read from TLS socket (remote: 127.14.164.62:40603): Connection reset by peer (error 104)
2026-08-10T07:24:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:06 dist-test-slave-x5fp krb5kdc[16689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:06 dist-test-slave-x5fp krb5kdc[16689](info): set up 2 sockets
Aug 10 07:24:06 dist-test-slave-x5fp krb5kdc[16689](info): commencing operation
krb5kdc: starting...
W20260810 07:24:07.920115 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:08 dist-test-slave-x5fp krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:08.058030 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36069
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:36069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:08.145493 16714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:08.145793 16714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:08.145839 16714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:08.147297 16714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:08.147380 16714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:08.147404 16714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:08.147431 16714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:08.147462 16714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:08.149129 16714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:36069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:08.149454 16714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:08.149712 16714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:08.154922 16726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.154981 16722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.155154 16714 server_base.cc:1034] running on GCE node
W20260810 07:24:08.155001 16724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.155507 16714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:08.155813 16714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:08.157013 16714 hybrid_clock.cc:648] HybridClock initialized: now 1786346648156988 us; error 47 us; skew 500 ppm
Aug 10 07:24:08 dist-test-slave-x5fp krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:08.160122 16714 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:08.160636 16714 webserver.cc:460] Webserver started at http://127.14.164.62:33313/ using document root <none> and password file <none>
I20260810 07:24:08.160908 16714 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:08.160981 16714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:08.161136 16714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:08.162287 16714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "dee89b0e0bc64eac90dbdf6715bf298f"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-x5fp"
server_key: "cf6cf051b38e5d967e6afb66fc79fab0"
server_key_iv: "6c1f04fce5d179312dda9da35b3601cc"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.162689 16714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "dee89b0e0bc64eac90dbdf6715bf298f"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-x5fp"
server_key: "cf6cf051b38e5d967e6afb66fc79fab0"
server_key_iv: "6c1f04fce5d179312dda9da35b3601cc"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.164080 16714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:24:08.164884 16737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.165093 16714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:08.165202 16714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "dee89b0e0bc64eac90dbdf6715bf298f"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-x5fp"
server_key: "cf6cf051b38e5d967e6afb66fc79fab0"
server_key_iv: "6c1f04fce5d179312dda9da35b3601cc"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.165285 16714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:08.168655 16714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:08.171139 16714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:08.171303 16714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:08.175640 16714 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:36069
I20260810 07:24:08.175758 16819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:36069 every 8 connection(s)
I20260810 07:24:08.176071 16714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:08.177268 16821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:08.179750 16821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f: Bootstrap starting.
I20260810 07:24:08.180392 16821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:08.180709 16821 log.cc:824] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:08.181543 16821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f: No bootstrap required, opened a new log
I20260810 07:24:08.183281 16821 raft_consensus.cc:348] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36069 } }
I20260810 07:24:08.183436 16821 raft_consensus.cc:374] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:08.183462 16821 raft_consensus.cc:729] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee89b0e0bc64eac90dbdf6715bf298f, State: Initialized, Role: FOLLOWER
I20260810 07:24:08.183653 16821 consensus_queue.cc:260] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36069 } }
I20260810 07:24:08.183751 16821 raft_consensus.cc:388] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:08.183779 16821 raft_consensus.cc:482] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:08.183856 16821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:08.184348 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16714
I20260810 07:24:08.184512 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:08.184757 14992 external_mini_cluster.cc:1428] Setting key e546da7b99a477bc5440d14cd653d09a
I20260810 07:24:08.185066 16821 raft_consensus.cc:504] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36069 } }
I20260810 07:24:08.185215 16821 leader_election.cc:302] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dee89b0e0bc64eac90dbdf6715bf298f; no voters:
I20260810 07:24:08.185406 16821 leader_election.cc:288] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:08.185432 16826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:08.185557 16826 raft_consensus.cc:686] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [term 1 LEADER]: Becoming Leader. State: Replica: dee89b0e0bc64eac90dbdf6715bf298f, State: Running, Role: LEADER
I20260810 07:24:08.185693 16821 sys_catalog.cc:564] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:08.185652 16826 consensus_queue.cc:237] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36069 } }
I20260810 07:24:08.186040 16826 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36069 } } }
I20260810 07:24:08.186124 16826 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:08.186285 16826 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dee89b0e0bc64eac90dbdf6715bf298f. Latest consensus state: current_term: 1 leader_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee89b0e0bc64eac90dbdf6715bf298f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36069 } } }
I20260810 07:24:08.186359 16826 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:08.187444 16834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:08.188009 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.185357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33171 (local address 127.14.164.62:36069)
0810 07:24:08.185645 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.185655 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.185715 (+ 60us) server_negotiation.cc:407] Connection header received
0810 07:24:08.185898 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.185905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.185939 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.186090 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.187301 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.187306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.187306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:08.187311 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.187365 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:08.187393 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.187714 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.187760 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.187807 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260810 07:24:08.188133 16834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:08.188994 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:08.190136 16834 catalog_manager.cc:1347] Generated new cluster ID: 389c6ffd5b534e7c9af1795a7524ab92
I20260810 07:24:08.190204 16834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:08.191982 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.190616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59527 (local address 127.14.164.62:36069)
0810 07:24:08.190701 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.190703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.190978 (+ 275us) server_negotiation.cc:407] Connection header received
0810 07:24:08.191008 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.191009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.191014 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.191071 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.191721 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.191723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.191723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:08.191727 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.191749 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:08.191753 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.191815 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.191895 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.191907 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:24:08.202776 16834 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:08.202977 16834 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:08.208105 16834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dee89b0e0bc64eac90dbdf6715bf298f: Generated new TSK 0
I20260810 07:24:08.208338 16834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:08.216403 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16714
2026-08-10T07:24:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:08 dist-test-slave-x5fp krb5kdc[16867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:08 dist-test-slave-x5fp krb5kdc[16867](info): set up 2 sockets
Aug 10 07:24:08 dist-test-slave-x5fp krb5kdc[16867](info): commencing operation
krb5kdc: starting...
W20260810 07:24:09.522404 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:09 dist-test-slave-x5fp krb5kdc[16867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:09.660619 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40203
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:42751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:40203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:09.746851 16889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:09.747192 16889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:09.747239 16889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:09.748523 16889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:09.748608 16889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:09.748634 16889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:09.748667 16889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:09.748696 16889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:09.750483 16889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:42751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:40203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:09.750833 16889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:09.751161 16889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:09.756511 16897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.756538 16896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.756855 16901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:09.756827 16889 server_base.cc:1034] running on GCE node
I20260810 07:24:09.757175 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:09.757527 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:09.758713 16889 hybrid_clock.cc:648] HybridClock initialized: now 1786346649758683 us; error 55 us; skew 500 ppm
Aug 10 07:24:09 dist-test-slave-x5fp krb5kdc[16867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346649, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:09.761461 16889 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:09.761934 16889 webserver.cc:460] Webserver started at http://127.14.164.62:44795/ using document root <none> and password file <none>
I20260810 07:24:09.762300 16889 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:09.762388 16889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:09.762538 16889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:09.763898 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "ac9ef2117fc5485d9b7d961ab4f368b0"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-x5fp"
server_key: "fb4e9846adbd6cac7092f0cbcf967b38"
server_key_iv: "7301acdb07b74498e45abd1760e5584a"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.764272 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "ac9ef2117fc5485d9b7d961ab4f368b0"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-x5fp"
server_key: "fb4e9846adbd6cac7092f0cbcf967b38"
server_key_iv: "7301acdb07b74498e45abd1760e5584a"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.765806 16889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 07:24:09.766677 16911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.766876 16889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:09.767005 16889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "ac9ef2117fc5485d9b7d961ab4f368b0"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-x5fp"
server_key: "fb4e9846adbd6cac7092f0cbcf967b38"
server_key_iv: "7301acdb07b74498e45abd1760e5584a"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.767072 16889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:09.780937 16889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:09.782994 16889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:09.783113 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:09.787299 16889 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:40203
I20260810 07:24:09.787330 16995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:40203 every 8 connection(s)
I20260810 07:24:09.787712 16889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:09.788650 16996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:09.791489 16996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0: Bootstrap starting.
I20260810 07:24:09.792251 16996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:09.792611 16996 log.cc:824] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:09.793516 16996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0: No bootstrap required, opened a new log
I20260810 07:24:09.795266 16996 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40203 } }
I20260810 07:24:09.795446 16996 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:09.795509 16996 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9ef2117fc5485d9b7d961ab4f368b0, State: Initialized, Role: FOLLOWER
I20260810 07:24:09.795680 16996 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40203 } }
I20260810 07:24:09.795770 16996 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:09.795825 16996 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:09.795876 16996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:09.796859 16996 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40203 } }
I20260810 07:24:09.796983 16996 leader_election.cc:302] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac9ef2117fc5485d9b7d961ab4f368b0; no voters:
I20260810 07:24:09.797194 16996 leader_election.cc:288] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:09.797238 17001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:09.797461 17001 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [term 1 LEADER]: Becoming Leader. State: Replica: ac9ef2117fc5485d9b7d961ab4f368b0, State: Running, Role: LEADER
I20260810 07:24:09.797483 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 16889
I20260810 07:24:09.797541 16996 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:09.797601 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:09.797554 17001 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40203 } }
I20260810 07:24:09.797811 14992 external_mini_cluster.cc:1428] Setting key d164b26c879746865ab8dae1e5bc5112
I20260810 07:24:09.797947 17003 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40203 } } }
I20260810 07:24:09.798022 17003 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:09.798177 17005 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac9ef2117fc5485d9b7d961ab4f368b0. Latest consensus state: current_term: 1 leader_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9ef2117fc5485d9b7d961ab4f368b0" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 40203 } } }
I20260810 07:24:09.798233 17005 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:09.798583 17013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:09.799129 17013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:09.800854 17013 catalog_manager.cc:1347] Generated new cluster ID: a754288b25184accaebd5e61f2549048
I20260810 07:24:09.800911 17013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:24:09 dist-test-slave-x5fp krb5kdc[16867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:09.810462 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.798522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35887 (local address 127.14.164.62:40203)
0810 07:24:09.798972 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.798983 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.799017 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:24:09.799147 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.799155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.799216 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.799371 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:09.800139 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.801601 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.802712 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.803312 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.803409 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.806501 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:09.806508 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:09.806508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:09.806526 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:09.808654 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:09.809072 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:09.809076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:09.809076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:09.809213 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:09.809472 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:09.809473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:09.809474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:09.809821 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:09.810000 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.810072 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.810218 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":77,"threads_started":1}
I20260810 07:24:09.811736 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:09.820932 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.813395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45335 (local address 127.14.164.62:40203)
0810 07:24:09.813430 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.813432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.813532 (+ 100us) server_negotiation.cc:407] Connection header received
0810 07:24:09.813622 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.813623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.813643 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.813671 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:09.814363 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.815139 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.816186 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.816519 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.816562 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.818089 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:09.818091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:09.818091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:09.818099 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:09.819621 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:09.820016 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:09.820017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:09.820017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:09.820071 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:09.820268 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:09.820269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:09.820269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:09.820405 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:09.820478 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.820692 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.820832 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 07:24:09.821352 17013 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:09.821992 17013 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:09.830744 17013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac9ef2117fc5485d9b7d961ab4f368b0: Generated new TSK 0
I20260810 07:24:09.830977 17013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:09.836513 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 16889
2026-08-10T07:24:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:09 dist-test-slave-x5fp krb5kdc[17047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:09 dist-test-slave-x5fp krb5kdc[17047](info): set up 2 sockets
Aug 10 07:24:09 dist-test-slave-x5fp krb5kdc[17047](info): commencing operation
krb5kdc: starting...
W20260810 07:24:11.105736 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:11 dist-test-slave-x5fp krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:11.244127 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36423
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:43825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:36423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:11.331413 17070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:11.331699 17070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:11.331744 17070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:11.333026 17070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:11.333096 17070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:11.333123 17070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:11.333156 17070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:11.333181 17070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:11.334774 17070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:43825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:36423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:36423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:11.335088 17070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:11.335338 17070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:11.340471 17082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:11.340478 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:11.340500 17081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:11.340844 17070 server_base.cc:1034] running on GCE node
I20260810 07:24:11.341179 17070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:11.341503 17070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:11.342682 17070 hybrid_clock.cc:648] HybridClock initialized: now 1786346651342678 us; error 60 us; skew 500 ppm
Aug 10 07:24:11 dist-test-slave-x5fp krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346651, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:11.350841 17070 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:11.351447 17070 webserver.cc:460] Webserver started at http://127.14.164.62:43921/ using document root <none> and password file <none>
I20260810 07:24:11.351715 17070 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:11.351794 17070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:11.351964 17070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:11.353250 17070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "6556f1947b2c43debf7f0339cc8ed2c2"
format_stamp: "Formatted at 2026-08-10 07:24:11 on dist-test-slave-x5fp"
server_key: "997fb69e72f358c053ef80cd60a16df3"
server_key_iv: "90aa9877ca1a27e72b997a8bcc9cd715"
server_key_version: "encryptionkey@0"
I20260810 07:24:11.353662 17070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "6556f1947b2c43debf7f0339cc8ed2c2"
format_stamp: "Formatted at 2026-08-10 07:24:11 on dist-test-slave-x5fp"
server_key: "997fb69e72f358c053ef80cd60a16df3"
server_key_iv: "90aa9877ca1a27e72b997a8bcc9cd715"
server_key_version: "encryptionkey@0"
I20260810 07:24:11.355423 17070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:11.356654 17101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:11.357054 17070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:11.357153 17070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "6556f1947b2c43debf7f0339cc8ed2c2"
format_stamp: "Formatted at 2026-08-10 07:24:11 on dist-test-slave-x5fp"
server_key: "997fb69e72f358c053ef80cd60a16df3"
server_key_iv: "90aa9877ca1a27e72b997a8bcc9cd715"
server_key_version: "encryptionkey@0"
I20260810 07:24:11.357219 17070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:11.378345 17070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:11.380928 17070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:11.381057 17070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:11.385349 17070 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:36423
I20260810 07:24:11.385388 17187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:36423 every 8 connection(s)
I20260810 07:24:11.385767 17070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:11.386726 17189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:11.389508 17189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2: Bootstrap starting.
I20260810 07:24:11.390095 17189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:11.390343 17189 log.cc:824] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:11.391099 17189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2: No bootstrap required, opened a new log
I20260810 07:24:11.391103 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17070
I20260810 07:24:11.391245 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:11.391463 14992 external_mini_cluster.cc:1428] Setting key b3559cb458d972ea79c5aae74a8b47d9
I20260810 07:24:11.393249 17189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36423 } }
I20260810 07:24:11.393432 17189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:11.393476 17189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6556f1947b2c43debf7f0339cc8ed2c2, State: Initialized, Role: FOLLOWER
I20260810 07:24:11.393654 17189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36423 } }
I20260810 07:24:11.393751 17189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:11.393776 17189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:11.393858 17189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:11.394459 17189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36423 } }
I20260810 07:24:11.394552 17189 leader_election.cc:302] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6556f1947b2c43debf7f0339cc8ed2c2; no voters:
I20260810 07:24:11.394757 17189 leader_election.cc:288] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:11.394781 17198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:11.394905 17198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [term 1 LEADER]: Becoming Leader. State: Replica: 6556f1947b2c43debf7f0339cc8ed2c2, State: Running, Role: LEADER
I20260810 07:24:11.395049 17198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36423 } }
I20260810 07:24:11.395049 17189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:11.396906 17200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6556f1947b2c43debf7f0339cc8ed2c2. Latest consensus state: current_term: 1 leader_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36423 } } }
I20260810 07:24:11.397002 17200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [sys.catalog]: This master's current role is: LEADER
W20260810 07:24:11.397219 17221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 07:24:11.397254 17199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556f1947b2c43debf7f0339cc8ed2c2" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 36423 } } }
I20260810 07:24:11.397375 17199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2 [sys.catalog]: This master's current role is: LEADER
W20260810 07:24:11.397303 17221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:11.398676 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:11.392041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50679 (local address 127.14.164.62:36423)
0810 07:24:11.392237 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:11.392245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:11.392273 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:24:11.393223 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:11.393230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:11.393284 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:11.393409 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:11.393709 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:11.395418 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:11.396328 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:11.397172 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:11.397629 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:11.397966 (+ 337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:11.397970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:11.397970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:11.398140 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:11.398172 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:11.398184 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:11.398240 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:11.398370 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:24:11.398452 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260810 07:24:11.399825 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:11.400911 17226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:11.401232 17226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:11.403403 17226 catalog_manager.cc:1347] Generated new cluster ID: 97e4a15c626d43db931d3a3e3b07a971
I20260810 07:24:11.403527 17226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:11.404923 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:11.401452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51909 (local address 127.14.164.62:36423)
0810 07:24:11.401491 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:11.401493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:11.401639 (+ 146us) server_negotiation.cc:407] Connection header received
0810 07:24:11.401699 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:11.401700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:11.401714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:11.401780 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:11.401957 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:11.402494 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:11.403504 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:11.403750 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:11.403841 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:11.404629 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:11.404630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:11.404631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:11.404636 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:11.404654 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:11.404659 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:11.404761 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:11.404800 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 07:24:11.404827 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 07:24:11.410185 17226 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:11.410929 17226 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:11.428743 17226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6556f1947b2c43debf7f0339cc8ed2c2: Generated new TSK 0
I20260810 07:24:11.429014 17226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:11.436542 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17070
2026-08-10T07:24:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:11 dist-test-slave-x5fp krb5kdc[17237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:11 dist-test-slave-x5fp krb5kdc[17237](info): set up 2 sockets
Aug 10 07:24:11 dist-test-slave-x5fp krb5kdc[17237](info): commencing operation
krb5kdc: starting...
W20260810 07:24:12.763254 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:12 dist-test-slave-x5fp krb5kdc[17237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:12.899794 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:37807
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:37807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:12.986138 17259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:12.986356 17259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:12.986380 17259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:12.987562 17259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:12.987607 17259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:12.987622 17259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:12.987634 17259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:12.987685 17259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:12.989224 17259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:37807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:37807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:12.989516 17259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:12.989765 17259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:12.994671 17267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.994807 17266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.994843 17269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.995198 17259 server_base.cc:1034] running on GCE node
I20260810 07:24:12.995514 17259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:12.995847 17259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:12.997053 17259 hybrid_clock.cc:648] HybridClock initialized: now 1786346652997037 us; error 45 us; skew 500 ppm
Aug 10 07:24:12 dist-test-slave-x5fp krb5kdc[17237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:12.999758 17259 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:13.000294 17259 webserver.cc:460] Webserver started at http://127.14.164.62:41741/ using document root <none> and password file <none>
I20260810 07:24:13.000535 17259 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:13.000604 17259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:13.000798 17259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:13.001917 17259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "8028458f4785410787b5394966dbe0aa"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-x5fp"
server_key: "dc3792d87927ae3130d38d4a7b8713b3"
server_key_iv: "7e3ec4c5a172bad4b15016d4af45cf05"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.002300 17259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "8028458f4785410787b5394966dbe0aa"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-x5fp"
server_key: "dc3792d87927ae3130d38d4a7b8713b3"
server_key_iv: "7e3ec4c5a172bad4b15016d4af45cf05"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.003691 17259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:24:13.004494 17281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.004679 17259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:24:13.004817 17259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "8028458f4785410787b5394966dbe0aa"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-x5fp"
server_key: "dc3792d87927ae3130d38d4a7b8713b3"
server_key_iv: "7e3ec4c5a172bad4b15016d4af45cf05"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.004905 17259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:13.008313 17259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:13.010866 17259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:13.011045 17259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:13.015158 17259 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:37807
I20260810 07:24:13.015190 17362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:37807 every 8 connection(s)
I20260810 07:24:13.015547 17259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:13.015789 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17259
I20260810 07:24:13.015933 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:13.016130 14992 external_mini_cluster.cc:1428] Setting key f61db8f2530d841b1af9a76051ad3999
I20260810 07:24:13.016821 17363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:13.018671 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.016708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40841 (local address 127.14.164.62:37807)
0810 07:24:13.016932 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.016943 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.016978 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:24:13.017431 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.017437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.017471 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.017590 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.018290 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.018293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.018294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.018298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.018337 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.018347 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.018404 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.018497 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.018530 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260810 07:24:13.019820 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:13.020289 17363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa: Bootstrap starting.
I20260810 07:24:13.020978 17363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:13.021315 17363 log.cc:824] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:13.022032 17363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa: No bootstrap required, opened a new log
I20260810 07:24:13.022491 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.021374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41089 (local address 127.14.164.62:37807)
0810 07:24:13.021419 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.021421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.021528 (+ 107us) server_negotiation.cc:407] Connection header received
0810 07:24:13.021601 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.021602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.021607 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.021652 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.022270 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.022272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.022272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.022276 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.022290 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.022294 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.022336 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.022392 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.022406 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:24:13.023952 17363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8028458f4785410787b5394966dbe0aa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37807 } }
I20260810 07:24:13.024117 17363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:13.024166 17363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8028458f4785410787b5394966dbe0aa, State: Initialized, Role: FOLLOWER
I20260810 07:24:13.024322 17363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8028458f4785410787b5394966dbe0aa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37807 } }
I20260810 07:24:13.024436 17363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:13.024483 17363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:13.024534 17363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:13.025192 17363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8028458f4785410787b5394966dbe0aa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37807 } }
I20260810 07:24:13.025321 17363 leader_election.cc:302] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8028458f4785410787b5394966dbe0aa; no voters:
I20260810 07:24:13.025522 17363 leader_election.cc:288] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:13.025609 17377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:13.025781 17377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [term 1 LEADER]: Becoming Leader. State: Replica: 8028458f4785410787b5394966dbe0aa, State: Running, Role: LEADER
I20260810 07:24:13.025820 17363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:13.025882 17377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8028458f4785410787b5394966dbe0aa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37807 } }
I20260810 07:24:13.026300 17380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8028458f4785410787b5394966dbe0aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8028458f4785410787b5394966dbe0aa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37807 } } }
I20260810 07:24:13.026331 17381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8028458f4785410787b5394966dbe0aa. Latest consensus state: current_term: 1 leader_uuid: "8028458f4785410787b5394966dbe0aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8028458f4785410787b5394966dbe0aa" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37807 } } }
I20260810 07:24:13.026459 17380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:13.026489 17381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:13.027066 17395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:13.027575 17395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:13.029117 17395 catalog_manager.cc:1347] Generated new cluster ID: f5231cdeff4c4a55b21ffe5ee4c78ccf
I20260810 07:24:13.029172 17395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:13.040685 17395 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:13.040946 17395 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:13.053316 17395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8028458f4785410787b5394966dbe0aa: Generated new TSK 0
I20260810 07:24:13.053573 17395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:13.059270 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17259
2026-08-10T07:24:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1614 ms)
[----------] 18 tests from AuthTokenIssuingTest (28931 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:13 dist-test-slave-x5fp krb5kdc[17416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:13 dist-test-slave-x5fp krb5kdc[17416](info): set up 2 sockets
Aug 10 07:24:13 dist-test-slave-x5fp krb5kdc[17416](info): commencing operation
krb5kdc: starting...
W20260810 07:24:14.340615 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:14.476121 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:37601
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:37601,127.14.164.61:39169,127.14.164.60:45297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:14.565362 17439 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:14.565791 17439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:14.565865 17439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:14.565920 17439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:14.567277 17439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:14.567354 17439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:14.567381 17439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:14.567414 17439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:14.567447 17439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:14.569303 17439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:37601,127.14.164.61:39169,127.14.164.60:45297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:37601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:14.569651 17439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:14.569940 17439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:14.575893 17452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:14.575881 17450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:14.575874 17449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:14.575834 17439 server_base.cc:1034] running on GCE node
I20260810 07:24:14.576795 17439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:14.577215 17439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:14.578457 17439 hybrid_clock.cc:648] HybridClock initialized: now 1786346654578427 us; error 42 us; skew 500 ppm
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:14.581506 17439 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:14.582067 17439 webserver.cc:460] Webserver started at http://127.14.164.62:43303/ using document root <none> and password file <none>
I20260810 07:24:14.582330 17439 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:14.582381 17439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:14.582485 17439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:14.583741 17439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "bc9ae65145344833abb6faa22bd81b5e"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "b7c18b37a0ef3da91d11ddf27e7920ec"
server_key_iv: "20b4ff46329e8552cd88eb195ffb8081"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.584111 17439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "bc9ae65145344833abb6faa22bd81b5e"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "b7c18b37a0ef3da91d11ddf27e7920ec"
server_key_iv: "20b4ff46329e8552cd88eb195ffb8081"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.585786 17439 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:14.586715 17460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:14.586947 17439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:14.587047 17439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "bc9ae65145344833abb6faa22bd81b5e"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "b7c18b37a0ef3da91d11ddf27e7920ec"
server_key_iv: "20b4ff46329e8552cd88eb195ffb8081"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.587118 17439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:14.598635 17439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:14.601194 17439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:14.601342 17439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:14.606251 17439 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:37601
I20260810 07:24:14.606303 17540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:37601 every 8 connection(s)
I20260810 07:24:14.606710 17439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:14.607916 17542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:14.609758 17542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:14.612941 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17439
I20260810 07:24:14.613070 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:14.613255 14992 external_mini_cluster.cc:1428] Setting key 9deba11d8ac51783373bf7d854530ac6
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:14.622092 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.610580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:37601 (local address 127.0.0.1:43204)
0810 07:24:14.610816 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.610865 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:14.611407 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:14.611736 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:14.611740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:14.612034 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:14.612196 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.612201 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.613480 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.613481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.614592 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.614596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.615336 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.615337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.615403 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.616001 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:14.616010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:14.617569 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:14.620128 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.620137 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.620140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.620376 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.620710 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.620711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.620711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.620915 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.621545 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:14.621553 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:14.621633 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 07:24:14.621728 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:24:14.621895 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":115,"threads_started":1}
I20260810 07:24:14.622188 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.610621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.14.164.62:37601)
0810 07:24:14.610861 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.610872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.611081 (+ 209us) server_negotiation.cc:407] Connection header received
0810 07:24:14.611576 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.611578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.611597 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.611698 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.612316 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.613364 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.614709 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.615199 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.615340 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.617678 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:14.617687 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:14.617688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:14.617696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:14.620005 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.620477 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.620481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.620481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.620609 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.621000 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.621002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.621002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.621431 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:14.621535 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:14.621801 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 07:24:14.622009 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":197,"thread_start_us":147,"threads_started":1}
I20260810 07:24:14.623288 17542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:14.623733 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.623583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48432)
0810 07:24:14.623639 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.623706 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.14.164.61@KRBTEST.COM; defaulting to no policy
W20260810 07:24:14.623929 17463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
I20260810 07:24:14.624497 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.624381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48442)
0810 07:24:14.624432 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.624465 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:24:14.624810 17542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:39169: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab.
Entry for principal kudu/127.14.164.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab.
I20260810 07:24:14.647794 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:39169
--webserver_interface=127.14.164.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab
--principal=kudu/127.14.164.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:37601,127.14.164.61:39169,127.14.164.60:45297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:14.674084 17542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } attempt: 1
I20260810 07:24:14.674671 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.674474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48446)
0810 07:24:14.674565 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.674624 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:24:14.675159 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.675045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48462)
0810 07:24:14.675095 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.675130 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:24:14.675318 17542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:39169: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
W20260810 07:24:14.740203 17550 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:14.740455 17550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:14.740475 17550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:14.740499 17550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:14.741855 17550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:14.741907 17550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:14.741923 17550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:14.741937 17550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:14.741951 17550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:14.743517 17542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } attempt: 2
I20260810 07:24:14.743685 17550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:37601,127.14.164.61:39169,127.14.164.60:45297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab
--principal=kudu/127.14.164.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:39169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.164.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:14.744037 17550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:14.744081 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.743931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48472)
0810 07:24:14.743992 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.744044 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 07:24:14.744349 17550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:24:14.744485 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.744371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48476)
0810 07:24:14.744422 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.744458 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:24:14.744633 17542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:39169: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39169: connect: Connection refused (error 111)
W20260810 07:24:14.749480 17560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:14.749481 17557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:14.749537 17556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:14.749897 17550 server_base.cc:1034] running on GCE node
I20260810 07:24:14.750165 17550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:14.750475 17550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:14.751655 17550 hybrid_clock.cc:648] HybridClock initialized: now 1786346654751618 us; error 54 us; skew 500 ppm
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:14.754591 17550 init.cc:377] Logged in from keytab as kudu/127.14.164.61@KRBTEST.COM (short username kudu)
I20260810 07:24:14.755115 17550 webserver.cc:460] Webserver started at http://127.14.164.61:35823/ using document root <none> and password file <none>
I20260810 07:24:14.755357 17550 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:14.755429 17550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:14.755578 17550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:14.756814 17550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data/instance:
uuid: "9d4a3903773d4412ac8ef20e983fffe5"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "f0b8d2e684dfac0517c3bb9527246835"
server_key_iv: "c433ac99f354e7bd6bf65218ac51de8a"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.757225 17550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal/instance:
uuid: "9d4a3903773d4412ac8ef20e983fffe5"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "f0b8d2e684dfac0517c3bb9527246835"
server_key_iv: "c433ac99f354e7bd6bf65218ac51de8a"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.758718 17550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:24:14.759589 17568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:14.759835 17550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:24:14.759974 17550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal
uuid: "9d4a3903773d4412ac8ef20e983fffe5"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "f0b8d2e684dfac0517c3bb9527246835"
server_key_iv: "c433ac99f354e7bd6bf65218ac51de8a"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.760058 17550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:14.798775 17550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:14.801026 17550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:14.801204 17550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:14.805312 17550 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.61:39169
I20260810 07:24:14.805378 17657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.61:39169 every 8 connection(s)
I20260810 07:24:14.805778 17550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
I20260810 07:24:14.806949 17660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:14.808702 17660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:14.815126 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17550
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:14.815213 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-1/wal/instance
I20260810 07:24:14.815407 14992 external_mini_cluster.cc:1428] Setting key da92f8ccaef5862f3de991bf0d0e421f
I20260810 07:24:14.819332 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.809646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.14.164.62:37601)
0810 07:24:14.809715 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.809718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.809974 (+ 256us) server_negotiation.cc:407] Connection header received
0810 07:24:14.810499 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.810500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.810515 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.810575 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.811263 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.811777 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.812952 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.813160 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.813219 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.815759 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:14.815760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:14.815761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:14.815770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:14.817649 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.818191 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.818192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.818192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.818252 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.818569 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.818570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.818570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.818681 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:14.818737 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:14.819098 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 07:24:14.819194 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:24:14.819424 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.809601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:37601 (local address 127.0.0.1:43212)
0810 07:24:14.809824 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.809872 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:14.810383 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:14.810663 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:14.810666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:14.810953 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:14.811169 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.811173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.811875 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.811876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.812872 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.812875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.813263 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.813264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.813407 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.814053 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:14.814074 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:14.815659 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:14.817738 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.817747 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.817750 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.818102 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.818322 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.818347 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.818347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.818499 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.818748 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:14.818756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:14.818941 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 07:24:14.819052 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:24:14.819219 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
I20260810 07:24:14.820070 17660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.164.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.60@KRBTEST.COM" created.
I20260810 07:24:14.828104 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.820357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48484)
0810 07:24:14.820428 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.820440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:14.820489 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:14.820848 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:14.820849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:14.821018 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:14.821075 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.821077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.821964 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.821965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.822498 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.822500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.823002 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.823002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.823070 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.823534 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:14.823539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:14.824793 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:14.826884 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.826885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.826885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.827079 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.827325 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.827326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.827326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.827380 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.827733 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:14.827734 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:14.827765 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 07:24:14.827844 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:24:14.827979 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 07:24:14.828172 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.820457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.14.164.61:39169)
0810 07:24:14.820646 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.820650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.820669 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:24:14.820695 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.820696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.820707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.820848 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.821166 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.821870 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.822586 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.822921 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.823006 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.824906 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:14.824914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:14.824915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:14.824921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:14.826799 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.827175 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.827178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.827178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.827261 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.827422 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.827422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.827422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.827624 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:14.827712 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:14.827880 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 07:24:14.828041 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
I20260810 07:24:14.829442 17660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:14.829821 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.829706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60338)
0810 07:24:14.829751 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.829799 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 07:24:14.829988 17575 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
I20260810 07:24:14.831439 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.831332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60352)
0810 07:24:14.831385 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.831411 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:24:14.831579 17660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45297: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
I20260810 07:24:14.835785 17542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } attempt: 3
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:24:14.843403 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.836122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48496)
0810 07:24:14.836173 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.836183 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:14.836241 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:14.836365 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:14.836366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:14.836528 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:14.836592 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.836594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.837474 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.837476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.838121 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.838124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.838590 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.838591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.838672 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.839128 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:14.839133 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:14.840422 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:14.842216 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.842217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.842218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.842472 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.842704 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.842705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.842705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.842790 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.843053 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:14.843054 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:14.843105 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 07:24:14.843192 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:24:14.843304 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:24:14.843453 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.836182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.14.164.61:39169)
0810 07:24:14.836246 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.836248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.836257 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:24:14.836302 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.836303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.836313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.836363 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.836674 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.837376 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.838204 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.838504 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.838590 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.840500 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:14.840501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:14.840502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:14.840510 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:14.842042 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.842556 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.842557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.842558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.842629 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.842857 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.842857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.842858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.842983 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:14.843060 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:14.843218 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 07:24:14.843350 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab.
Entry for principal kudu/127.14.164.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab.
I20260810 07:24:14.843897 17542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:14.844259 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.844156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60356)
0810 07:24:14.844205 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.844233 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:24:14.844388 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:45297
--webserver_interface=127.14.164.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab
--principal=kudu/127.14.164.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:37601,127.14.164.61:39169,127.14.164.60:45297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:14.844652 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.844547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60364)
0810 07:24:14.844594 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.844619 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:24:14.844802 17542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45297: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
I20260810 07:24:14.876001 17542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } attempt: 1
I20260810 07:24:14.876657 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.876466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60378)
0810 07:24:14.876569 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.876619 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 07:24:14.877224 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.877091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60394)
0810 07:24:14.877160 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.877197 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:24:14.877369 17542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45297: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
I20260810 07:24:14.880856 17660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } attempt: 1
I20260810 07:24:14.881299 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.881160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60406)
0810 07:24:14.881230 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.881270 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:24:14.881644 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.881550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60410)
0810 07:24:14.881595 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.881620 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:24:14.881732 17660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45297: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
W20260810 07:24:14.931272 17670 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:14.931556 17670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:14.931587 17670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:14.931636 17670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:14.932853 17670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:14.932921 17670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:14.932947 17670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:14.932967 17670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:14.932999 17670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:14.934522 17670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:37601,127.14.164.61:39169,127.14.164.60:45297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab
--principal=kudu/127.14.164.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:45297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.164.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:14.934815 17670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:14.935067 17670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:14.940168 17685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:14.940184 17688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:14.940167 17683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:14.940500 17670 server_base.cc:1034] running on GCE node
I20260810 07:24:14.940811 17670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:14.941143 17670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:14.942328 17670 hybrid_clock.cc:648] HybridClock initialized: now 1786346654942288 us; error 66 us; skew 500 ppm
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:14.945001 17670 init.cc:377] Logged in from keytab as kudu/127.14.164.60@KRBTEST.COM (short username kudu)
I20260810 07:24:14.945469 17670 webserver.cc:460] Webserver started at http://127.14.164.60:38821/ using document root <none> and password file <none>
I20260810 07:24:14.945715 17670 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:14.945804 17670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:14.945966 17670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:14.947052 17670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data/instance:
uuid: "5fe94b3480cd406ea0f208fd6d808f09"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "f92114d3b84c0c937532f0f77f719023"
server_key_iv: "d4d9bbef8a67ac9c9b00ac104a5f552b"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.947371 17670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal/instance:
uuid: "5fe94b3480cd406ea0f208fd6d808f09"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "f92114d3b84c0c937532f0f77f719023"
server_key_iv: "d4d9bbef8a67ac9c9b00ac104a5f552b"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.948761 17670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:24:14.949625 17700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:14.949898 17670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:14.949872 17660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } attempt: 2
I20260810 07:24:14.949995 17670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal
uuid: "5fe94b3480cd406ea0f208fd6d808f09"
format_stamp: "Formatted at 2026-08-10 07:24:14 on dist-test-slave-x5fp"
server_key: "f92114d3b84c0c937532f0f77f719023"
server_key_iv: "d4d9bbef8a67ac9c9b00ac104a5f552b"
server_key_version: "encryptionkey@0"
I20260810 07:24:14.950083 17670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:14.950278 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.950145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60418)
0810 07:24:14.950204 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.950244 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 07:24:14.950681 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.950563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60422)
0810 07:24:14.950621 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.950649 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:24:14.950807 17660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45297: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
I20260810 07:24:14.952541 17542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } attempt: 2
I20260810 07:24:14.952991 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.952842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60432)
0810 07:24:14.952909 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.952951 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:24:14.953471 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.953358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60438)
0810 07:24:14.953415 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.953447 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:24:14.953610 17542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45297: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45297: connect: Connection refused (error 111)
I20260810 07:24:14.977834 17670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:14.979907 17670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:14.980073 17670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:14.984498 17670 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.60:45297
I20260810 07:24:14.984565 17780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.60:45297 every 8 connection(s)
I20260810 07:24:14.984978 17670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
I20260810 07:24:14.986066 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:14.987699 17784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:14.990759 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 17670
I20260810 07:24:14.990867 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346363752364-14992-0/minicluster-data/master-2/wal/instance
I20260810 07:24:14.991063 14992 external_mini_cluster.cc:1428] Setting key d30b3ef9926626b95f18dadd555bba09
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:14.997900 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.988510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.14.164.62:37601)
0810 07:24:14.988578 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.988580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.988752 (+ 172us) server_negotiation.cc:407] Connection header received
0810 07:24:14.989272 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.989273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.989285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.989355 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.990023 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.990549 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.992087 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.992275 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.992330 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.994705 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:14.994707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:14.994707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:14.994715 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:14.996258 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.996702 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.996703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.996704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.996790 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.997101 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.997102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.997102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.997219 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:14.997302 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:14.997689 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 07:24:14.997784 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:24:14.998090 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.988395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:37601 (local address 127.0.0.1:43226)
0810 07:24:14.988582 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.988626 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:14.989140 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:14.989434 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:14.989437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:14.989695 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:14.989890 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.989893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.990647 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.990649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.992000 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.992004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.992386 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.992388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:14.992566 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.993157 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:14.993166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:14.994614 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:14.996355 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.996363 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.996365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.996611 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.996864 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:14.996865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:14.996865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:14.996973 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:14.997309 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:14.997317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:14.997503 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 07:24:14.997630 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:24:14.997806 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260810 07:24:14.998703 17784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:15.000554 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.991484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.14.164.60:45297)
0810 07:24:14.991667 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.991671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.991717 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:24:14.991797 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.991798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.991815 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.991882 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.992844 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.993535 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.994345 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.994673 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.994750 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.997387 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:14.997393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:14.997393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:14.997400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:14.999344 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.999724 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.999726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.999727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:14.999782 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:14.999998 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:14.999999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:14.999999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.000129 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.000230 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.000367 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.000447 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
Aug 10 07:24:14 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
Aug 10 07:24:15 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:15.007762 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.999019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39169 (local address 127.0.0.1:48502)
0810 07:24:14.999158 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.999169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:14.999218 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:14.999343 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:14.999343 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:14.999489 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:14.999522 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:14.999523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.000340 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.000341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.001409 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.001413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.001901 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.001902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.002007 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.002615 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:15.002620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:15.004036 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:15.006558 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.006560 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.006560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.006795 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.007084 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.007086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.007086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.007160 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.007449 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:15.007450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:15.007505 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 07:24:15.007583 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:24:15.007681 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 07:24:15.007894 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.999110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.14.164.61:39169)
0810 07:24:14.999156 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.999158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.999204 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:24:14.999280 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.999281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.999290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.999338 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:14.999598 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.000252 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.001539 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.001813 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.001892 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.004131 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.004132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.004133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.004140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.006457 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.006927 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.006929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.006929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.006999 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.007228 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.007229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.007229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.007370 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.007431 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.007638 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.007743 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:24:15.008453 17784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:15.011523 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.001841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.14.164.62:37601)
0810 07:24:15.001909 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.001911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.001919 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:24:15.001942 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.001943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.001954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.002142 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.002952 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.003484 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.004555 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.004901 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.004987 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.007636 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.007638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.007638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.007647 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.010329 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.010684 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.010685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.010686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.010755 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.010952 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.010952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.010953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.011086 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.011205 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.011316 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.011420 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:24:15 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:15.016093 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.008780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60456)
0810 07:24:15.008914 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:15.008926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:15.009007 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:15.009181 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:15.009182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:15.009362 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:15.009416 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.009419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.010166 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.010167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.010928 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.010931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.011393 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.011395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.011457 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.011836 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:15.011842 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:15.013164 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:15.015000 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.015001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.015002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.015218 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.015472 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.015473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.015473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.015530 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.015814 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:15.015815 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:15.015868 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:24:15.015942 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:24:15.016023 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 07:24:15.016244 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.008874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.14.164.60:45297)
0810 07:24:15.008918 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.008920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.008961 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:24:15.009085 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.009086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.009101 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.009149 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.009500 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.010068 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.011005 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.011311 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.011374 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.013265 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.013267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.013267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.013275 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.014886 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.015309 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.015309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.015310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.015363 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.015598 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.015599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.015600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.015724 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.015809 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.016035 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.016122 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 10 07:24:15 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:24:15.018229 17784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: Bootstrap starting.
I20260810 07:24:15.019424 17784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:15.019780 17784 log.cc:824] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:15.020820 17784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: No bootstrap required, opened a new log
I20260810 07:24:15.021428 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.012516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.14.164.61:39169)
0810 07:24:15.012581 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.012584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.012591 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:24:15.012613 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.012613 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.012623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.012698 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.013309 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.013917 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.015001 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.015335 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.015394 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.018210 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.018212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.018212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.018221 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.019948 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.020436 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.020437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.020437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.020503 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.020835 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.020837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.020837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.020986 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.021178 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.021230 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.021320 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:24:15.022987 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:15.022914 17784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.023063 17784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe94b3480cd406ea0f208fd6d808f09, State: Initialized, Role: FOLLOWER
I20260810 07:24:15.023175 17784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.023526 17791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.023528 17784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:15.023679 17791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:15.024197 17493 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bc9ae65145344833abb6faa22bd81b5e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43236
W20260810 07:24:15.026495 17806 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:15.026564 17806 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:15.026593 17670 master_runner.cc:186] Error getting master registration for 127.14.164.62:37601: OK, instance_id { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" instance_seqno: 1786346654604662 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:15.029798 17670 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:15.041949 17660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } attempt: 3
Aug 10 07:24:15 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:15.050619 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.042492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.14.164.60:45297)
0810 07:24:15.042541 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.042543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.042555 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:24:15.042590 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.042591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.042602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.042663 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.043113 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.043874 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.044923 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.045243 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.045315 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.047525 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.047527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.047527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.047535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.049214 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.049618 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.049619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.049619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.049674 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.049944 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.049945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.049945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.050116 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.050225 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.050417 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.050515 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:24:15.050619 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.042365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60462)
0810 07:24:15.042453 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:15.042469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:15.042536 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:15.042699 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:15.042700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:15.042957 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:15.043020 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.043022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.043975 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.043976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.044810 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.044812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.045324 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.045325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.045384 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.045921 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:15.045927 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:15.047410 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:15.049295 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.049296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.049297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.049539 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.049767 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.049768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.049769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.049846 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.050215 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:15.050216 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:15.050280 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 07:24:15.050388 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:24:15.050509 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:24:15.052780 17542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } attempt: 3
I20260810 07:24:15.052829 17660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: Bootstrap starting.
I20260810 07:24:15.053689 17660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:15.054018 17660 log.cc:824] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:15.055022 17660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: No bootstrap required, opened a new log
Aug 10 07:24:15 dist-test-slave-x5fp krb5kdc[17416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:15.057168 17660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.057320 17660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d4a3903773d4412ac8ef20e983fffe5, State: Initialized, Role: FOLLOWER
I20260810 07:24:15.057421 17660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.057811 17660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:15.057814 17814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.058183 17814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:24:15.060307 17834 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:15.060364 17834 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:15.060869 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.053101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45297 (local address 127.0.0.1:60476)
0810 07:24:15.053154 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:15.053167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:15.053213 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:15.053411 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:15.053412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:15.053598 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:15.053652 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.053654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.054487 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.054488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.055089 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:15.055093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.055462 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.055463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:15.055524 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.056062 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:15.056086 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:15.057632 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:15.059686 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.059687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.059688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.059931 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.060142 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:15.060143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:15.060143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:15.060207 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:15.060501 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:15.060502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:15.060557 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 07:24:15.060623 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:24:15.060753 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:24:15.061172 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.053226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.14.164.60:45297)
0810 07:24:15.053299 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.053301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.053307 (+ 6us) server_negotiation.cc:407] Connection header received
0810 07:24:15.053324 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.053325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.053332 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.053402 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.053729 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.054369 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.055197 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.055386 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.055460 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.057748 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.057749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.057750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.057757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.059605 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.060001 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.060003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.060003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.060083 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.060260 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.060261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.060262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.060433 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.060511 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.060997 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.061087 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:24:15.059829 17550 master_runner.cc:186] Error getting master registration for 127.14.164.62:37601: OK, instance_id { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" instance_seqno: 1786346654604662 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:15.062762 17550 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:15.064478 17542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e: Bootstrap starting.
I20260810 07:24:15.065336 17542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:15.065668 17542 log.cc:824] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:15.066591 17542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e: No bootstrap required, opened a new log
I20260810 07:24:15.068167 17542 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.068323 17542 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9ae65145344833abb6faa22bd81b5e, State: Initialized, Role: FOLLOWER
I20260810 07:24:15.068413 17542 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.068786 17542 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:15.068827 17493 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:24:15.068915 17493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:15.069801 17493 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.070837 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc9ae65145344833abb6faa22bd81b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9d4a3903773d4412ac8ef20e983fffe5"
I20260810 07:24:15.070997 17616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:15.072333 17616 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9ae65145344833abb6faa22bd81b5e in term 1.
I20260810 07:24:15.072644 17745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc9ae65145344833abb6faa22bd81b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5fe94b3480cd406ea0f208fd6d808f09"
I20260810 07:24:15.072814 17745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:15.073757 17745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9ae65145344833abb6faa22bd81b5e in term 1.
I20260810 07:24:15.070130 17493 leader_election.cc:288] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [CANDIDATE]: Term 1 election: Requested vote from peers 9d4a3903773d4412ac8ef20e983fffe5 (127.14.164.61:39169), 5fe94b3480cd406ea0f208fd6d808f09 (127.14.164.60:45297)
W20260810 07:24:15.071084 17858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:15.076182 17858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:15.072659 17463 leader_election.cc:302] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d4a3903773d4412ac8ef20e983fffe5, bc9ae65145344833abb6faa22bd81b5e; no voters:
I20260810 07:24:15.078189 17840 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.078385 17840 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:15.078490 17840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:15.078676 17840 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [term 1 LEADER]: Becoming Leader. State: Replica: bc9ae65145344833abb6faa22bd81b5e, State: Running, Role: LEADER
I20260810 07:24:15.078810 17840 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } }
I20260810 07:24:15.079000 17439 master_runner.cc:263] No leader master found from master bc9ae65145344833abb6faa22bd81b5e
I20260810 07:24:15.079062 17439 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:15.079435 17840 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc9ae65145344833abb6faa22bd81b5e. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.079577 17840 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:15.079810 17868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:15.080052 17868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:15.082089 17745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [term 1 FOLLOWER]: Refusing update from remote peer bc9ae65145344833abb6faa22bd81b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:15.082489 17866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:15.083927 17791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc9ae65145344833abb6faa22bd81b5e. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.084026 17791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:15.084522 17616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [term 1 FOLLOWER]: Refusing update from remote peer bc9ae65145344833abb6faa22bd81b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:15.084920 17840 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:15.086283 17814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc9ae65145344833abb6faa22bd81b5e. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.086373 17814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:15.088099 17840 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.088182 17840 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:15.088708 17791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.088822 17791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:15.089124 17869 mvcc.cc:205] Tried to move back new op lower bound from 7316875899213783040 to 7316875899204214784. Current Snapshot: MvccSnapshot[applied={T|T < 7316875899213783040}]
I20260810 07:24:15.089382 17880 mvcc.cc:205] Tried to move back new op lower bound from 7316875899213783040 to 7316875899204214784. Current Snapshot: MvccSnapshot[applied={T|T < 7316875899213783040}]
I20260810 07:24:15.093649 17868 catalog_manager.cc:1347] Generated new cluster ID: 8f36a326583e4c8ab98d4cfa1cc06312
I20260810 07:24:15.093715 17868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:15.103150 17866 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.103271 17866 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:15.105727 17814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc9ae65145344833abb6faa22bd81b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ae65145344833abb6faa22bd81b5e" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 37601 } } peers { permanent_uuid: "9d4a3903773d4412ac8ef20e983fffe5" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39169 } } peers { permanent_uuid: "5fe94b3480cd406ea0f208fd6d808f09" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45297 } } }
I20260810 07:24:15.105839 17814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:15.105958 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.081014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.14.164.62:37601)
0810 07:24:15.081058 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.081061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.088892 (+ 7831us) server_negotiation.cc:407] Connection header received
0810 07:24:15.089005 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.089006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.089017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.089048 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.090885 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.091667 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.094578 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.094888 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.094933 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.099720 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.099722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.099722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.099731 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.101512 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.104311 (+ 2799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.104312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.104312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.104378 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.104886 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.104887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.104887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.105023 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.105107 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.105765 (+ 658us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.105860 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:24:15.106339 17882 mvcc.cc:205] Tried to move back new op lower bound from 7316875899213783040 to 7316875899204214784. Current Snapshot: MvccSnapshot[applied={T|T < 7316875899213783040}]
I20260810 07:24:15.106792 17868 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:15.107594 17868 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:15.107686 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.081436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.14.164.60:45297)
0810 07:24:15.081472 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.081474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.090020 (+ 8546us) server_negotiation.cc:407] Connection header received
0810 07:24:15.090117 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.090118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.090128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.090158 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.092839 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.093571 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.097633 (+ 4062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.097943 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.097988 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.101933 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.101935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.101935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.101943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.103885 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.104609 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.104610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.104611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.104664 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.105298 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.105299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.105300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.105451 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.105561 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.107492 (+ 1931us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.107587 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:24:15.117687 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.081241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.14.164.61:39169)
0810 07:24:15.081279 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.081281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.089646 (+ 8365us) server_negotiation.cc:407] Connection header received
0810 07:24:15.089752 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.089753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.089765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.089798 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.091195 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.091973 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.107858 (+ 15885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.108195 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.108241 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.114116 (+ 5875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.114118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.114119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.114128 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.115853 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.116299 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.116300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.116300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.116358 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.116556 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.116557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.116557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.116698 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.116832 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.117494 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.117581 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:24:15.126205 17868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc9ae65145344833abb6faa22bd81b5e: Generated new TSK 0
I20260810 07:24:15.126533 17868 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:24:15.152501 17789 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.150685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.14.164.60:45297)
0810 07:24:15.150732 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.150735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.151734 (+ 999us) server_negotiation.cc:407] Connection header received
0810 07:24:15.151860 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.151861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.151872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.151905 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.152399 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60486: BlockingRecv error: recv got EOF from 127.0.0.1:60486 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 07:24:15.153580 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.151121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48530 (local address 127.14.164.61:39169)
0810 07:24:15.151162 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.151164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.152982 (+ 1818us) server_negotiation.cc:407] Connection header received
0810 07:24:15.153083 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.153084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.153096 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.153126 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.153515 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48530: BlockingRecv error: recv got EOF from 127.0.0.1:48530 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:24:15.154700 17546 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.151455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.14.164.62:37601)
0810 07:24:15.151497 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.151499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.153996 (+ 2497us) server_negotiation.cc:407] Connection header received
0810 07:24:15.154110 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.154112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.154123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.154154 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.154632 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43268: BlockingRecv error: recv got EOF from 127.0.0.1:43268 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 07:24:15.161921 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.161068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.14.164.61:39169)
0810 07:24:15.161120 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.161123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.161267 (+ 144us) server_negotiation.cc:407] Connection header received
0810 07:24:15.161397 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.161398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.161417 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.161458 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.161860 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48540: BlockingRecv error: recv got EOF from 127.0.0.1:48540 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 07:24:15.173440 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.172420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.14.164.61:39169)
0810 07:24:15.172485 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.172487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.172830 (+ 343us) server_negotiation.cc:407] Connection header received
0810 07:24:15.172938 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.172939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.172952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.172982 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.173380 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48542: BlockingRecv error: recv got EOF from 127.0.0.1:48542 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:24:15.177847 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.176944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.14.164.61:39169)
0810 07:24:15.176984 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.176986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.177254 (+ 268us) server_negotiation.cc:407] Connection header received
0810 07:24:15.177363 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.177364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.177376 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.177407 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.177787 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48556: BlockingRecv error: recv got EOF from 127.0.0.1:48556 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 07:24:15.184232 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.183284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.14.164.61:39169)
0810 07:24:15.183325 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.183327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.183611 (+ 284us) server_negotiation.cc:407] Connection header received
0810 07:24:15.183719 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.183720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.183733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.183778 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.184175 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48560: BlockingRecv error: recv got EOF from 127.0.0.1:48560 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 07:24:15.194725 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.193736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.14.164.61:39169)
0810 07:24:15.193797 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.193799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.194112 (+ 313us) server_negotiation.cc:407] Connection header received
0810 07:24:15.194225 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.194226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.194239 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.194271 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.194661 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48574: BlockingRecv error: recv got EOF from 127.0.0.1:48574 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 07:24:15.213346 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.212254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.14.164.61:39169)
0810 07:24:15.212304 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.212306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.212635 (+ 329us) server_negotiation.cc:407] Connection header received
0810 07:24:15.212800 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.212801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.212816 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.212849 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.213278 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48584: BlockingRecv error: recv got EOF from 127.0.0.1:48584 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 07:24:15.247834 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.247008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.14.164.61:39169)
0810 07:24:15.247062 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.247065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.247219 (+ 154us) server_negotiation.cc:407] Connection header received
0810 07:24:15.247298 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.247299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.247313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.247390 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.247772 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48586: BlockingRecv error: recv got EOF from 127.0.0.1:48586 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 07:24:15.314086 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.313312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.14.164.61:39169)
0810 07:24:15.313419 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.313421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.313499 (+ 78us) server_negotiation.cc:407] Connection header received
0810 07:24:15.313557 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.313558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.313570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.313624 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.314017 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48588: BlockingRecv error: recv got EOF from 127.0.0.1:48588 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 07:24:15.444322 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.443573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.14.164.61:39169)
0810 07:24:15.443675 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.443677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.443745 (+ 68us) server_negotiation.cc:407] Connection header received
0810 07:24:15.443794 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.443795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.443807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.443867 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.444228 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48592: BlockingRecv error: recv got EOF from 127.0.0.1:48592 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 07:24:15.702888 17667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:15.702016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.14.164.61:39169)
0810 07:24:15.702129 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.702131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.702162 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:24:15.702263 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.702264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.702276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.702345 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.702787 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48596: BlockingRecv error: recv got EOF from 127.0.0.1:48596 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 07:24:16.027118 17806 catalog_manager.cc:1259] Loaded cluster ID: 8f36a326583e4c8ab98d4cfa1cc06312
I20260810 07:24:16.027280 17806 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: loading cluster ID for follower catalog manager: success
I20260810 07:24:16.028242 17806 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: acquiring CA information for follower catalog manager: success
I20260810 07:24:16.028611 17806 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5fe94b3480cd406ea0f208fd6d808f09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:16.060897 17834 catalog_manager.cc:1259] Loaded cluster ID: 8f36a326583e4c8ab98d4cfa1cc06312
I20260810 07:24:16.060988 17834 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: loading cluster ID for follower catalog manager: success
I20260810 07:24:16.061913 17834 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: acquiring CA information for follower catalog manager: success
I20260810 07:24:16.062251 17834 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d4a3903773d4412ac8ef20e983fffe5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:16.219340 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:16.216353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.14.164.61:39169)
0810 07:24:16.216474 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:16.216476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:16.216510 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:24:16.216587 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:16.216588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:16.216646 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:16.216792 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:16.216980 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.217476 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.218362 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.218566 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.218637 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:16.218839 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:16.218959 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:16.219029 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:16.219161 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 07:24:16.219211 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260810 07:24:16.224026 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:16.221669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.14.164.60:45297)
0810 07:24:16.221799 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:16.221802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:16.221830 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:24:16.221867 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:16.221868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:16.221906 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:16.221972 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:16.222094 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.222573 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.223313 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.223527 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.223581 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:16.223718 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:16.223830 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:16.223896 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:16.223914 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 07:24:16.223932 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260810 07:24:16.228765 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:16.225823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.14.164.60:45297)
0810 07:24:16.225861 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:16.225863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:16.226107 (+ 244us) server_negotiation.cc:407] Connection header received
0810 07:24:16.226190 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:16.226192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:16.226203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:16.226249 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:16.226526 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.226988 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.227934 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.228223 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.228287 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:16.228421 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:16.228452 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:16.228503 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:16.228625 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:24:16.228650 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:24:16.229151 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:16.225787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.14.164.61:39169)
0810 07:24:16.225859 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:16.225862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:16.226082 (+ 220us) server_negotiation.cc:407] Connection header received
0810 07:24:16.226144 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:16.226145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:16.226158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:16.226209 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:16.226526 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.227253 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.228225 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.228552 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.228612 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:16.228843 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:16.228875 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:16.228904 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:16.229033 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:24:16.229058 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:24:16.229266 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:16.225842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.14.164.62:37601)
0810 07:24:16.225976 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:16.225978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:16.225987 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:24:16.226077 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:16.226079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:16.226135 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:16.226218 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:16.226501 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.227280 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.228151 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:16.228477 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:16.228565 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:16.228794 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:16.228914 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:16.228976 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:16.229154 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 07:24:16.229183 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260810 07:24:16.231026 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17439
I20260810 07:24:16.246389 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17550
I20260810 07:24:16.259526 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 17670
2026-08-10T07:24:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3202 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:16 dist-test-slave-x5fp krb5kdc[18025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:16 dist-test-slave-x5fp krb5kdc[18025](info): set up 2 sockets
Aug 10 07:24:16 dist-test-slave-x5fp krb5kdc[18025](info): commencing operation
krb5kdc: starting...
W20260810 07:24:17.587927 14992 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/test-user.keytab.
Aug 10 07:24:17 dist-test-slave-x5fp krb5kdc[18025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.164.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
Entry for principal kudu/127.14.164.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab.
I20260810 07:24:17.724020 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:46475
--webserver_interface=127.14.164.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:46475,127.14.164.61:39103,127.14.164.60:45893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:17.811515 18040 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:17.811774 18040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:17.811805 18040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:17.811848 18040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:17.813103 18040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:17.813169 18040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:17.813194 18040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:17.813227 18040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:17.813259 18040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:17.814823 18040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:46475,127.14.164.61:39103,127.14.164.60:45893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.62.keytab
--principal=kudu/127.14.164.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.62:46475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.164.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:17.815106 18040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:17.815366 18040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:17.820536 18052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:17.820542 18060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:17.820544 18057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:17.820848 18040 server_base.cc:1034] running on GCE node
I20260810 07:24:17.821254 18040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:17.821615 18040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:17.822794 18040 hybrid_clock.cc:648] HybridClock initialized: now 1786346657822769 us; error 40 us; skew 500 ppm
Aug 10 07:24:17 dist-test-slave-x5fp krb5kdc[18025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:17.831851 18040 init.cc:377] Logged in from keytab as kudu/127.14.164.62@KRBTEST.COM (short username kudu)
I20260810 07:24:17.832404 18040 webserver.cc:460] Webserver started at http://127.14.164.62:40213/ using document root <none> and password file <none>
I20260810 07:24:17.832697 18040 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:17.832828 18040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:17.832993 18040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:17.834144 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data/instance:
uuid: "168c43fb9b7c41a8bc05fb2ef402a70f"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-x5fp"
server_key: "bd99513865a1770d79a82f9f969a8fa9"
server_key_iv: "079f4a71d9c4212555da8c1be9d6cc29"
server_key_version: "encryptionkey@0"
I20260810 07:24:17.834555 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal/instance:
uuid: "168c43fb9b7c41a8bc05fb2ef402a70f"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-x5fp"
server_key: "bd99513865a1770d79a82f9f969a8fa9"
server_key_iv: "079f4a71d9c4212555da8c1be9d6cc29"
server_key_version: "encryptionkey@0"
I20260810 07:24:17.836010 18040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:24:17.837216 18072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:17.837456 18040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:24:17.837565 18040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal
uuid: "168c43fb9b7c41a8bc05fb2ef402a70f"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-x5fp"
server_key: "bd99513865a1770d79a82f9f969a8fa9"
server_key_iv: "079f4a71d9c4212555da8c1be9d6cc29"
server_key_version: "encryptionkey@0"
I20260810 07:24:17.837644 18040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:17.845382 18040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:17.847460 18040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:17.847579 18040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:17.851754 18040 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.62:46475
I20260810 07:24:17.851840 18168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.62:46475 every 8 connection(s)
I20260810 07:24:17.852209 18040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:17.853446 18169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:17.855067 18169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:17.860682 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18040
I20260810 07:24:17.860824 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-0/wal/instance
I20260810 07:24:17.861073 14992 external_mini_cluster.cc:1428] Setting key 97b37b124f8b5d27538205b5bcb0a583
Aug 10 07:24:17 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:17.865636 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.855837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:46475 (local address 127.0.0.1:52046)
0810 07:24:17.856026 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.856071 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:17.856538 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:17.856854 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:17.856857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:17.857111 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:17.857271 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:17.857275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.858326 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.858327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:17.859003 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:17.859005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.859510 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.859511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:17.859566 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:17.860183 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:17.860191 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:17.862090 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:17.863979 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:17.863988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:17.863992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:17.864297 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:17.864591 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:17.864592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:17.864592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:17.864675 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:17.865153 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:17.865159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:17.865246 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 07:24:17.865328 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:24:17.865464 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
I20260810 07:24:17.865734 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.855883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.14.164.62:46475)
0810 07:24:17.856075 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:17.856081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:17.856175 (+ 94us) server_negotiation.cc:407] Connection header received
0810 07:24:17.856682 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:17.856684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:17.856702 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:17.856851 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:17.857393 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.858221 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.859086 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.859408 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.859488 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:17.862186 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:17.862193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:17.862194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:17.862200 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:17.863885 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:17.864398 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:17.864402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:17.864402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:17.864518 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:17.864753 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:17.864754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:17.864754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:17.865079 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:17.865166 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:17.865394 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 07:24:17.865555 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260810 07:24:17.866518 18169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:17.866921 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.866767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50510)
0810 07:24:17.866822 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.866884 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:24:17.867082 18076 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
I20260810 07:24:17.867596 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.867504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50520)
0810 07:24:17.867551 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.867577 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:24:17.867705 18169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:39103: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.164.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab.
Entry for principal kudu/127.14.164.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab.
I20260810 07:24:17.892814 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:39103
--webserver_interface=127.14.164.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab
--principal=kudu/127.14.164.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:46475,127.14.164.61:39103,127.14.164.60:45893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:17.916956 18169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } attempt: 1
I20260810 07:24:17.917582 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.917431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50526)
0810 07:24:17.917502 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.917547 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:24:17.917975 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.917864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50532)
0810 07:24:17.917921 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.917955 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 07:24:17.918095 18169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:39103: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
W20260810 07:24:17.983920 18178 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:17.984189 18178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:17.984220 18178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:17.984261 18178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:17.985522 18178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:17.985592 18178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:17.985617 18178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:17.985646 18178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:17.985675 18178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:17.986259 18169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } attempt: 2
I20260810 07:24:17.986764 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.986593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50536)
0810 07:24:17.986670 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.986722 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:24:17.987233 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.987118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50548)
0810 07:24:17.987170 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:17.987201 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:24:17.987375 18169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.61:39103: Network error: Client connection negotiation failed: client connection to 127.14.164.61:39103: connect: Connection refused (error 111)
I20260810 07:24:17.987658 18178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:46475,127.14.164.61:39103,127.14.164.60:45893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.61.keytab
--principal=kudu/127.14.164.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.61:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.164.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:17.987931 18178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:17.988185 18178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:17.993189 18186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:17.993220 18185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:17.993176 18188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:17.993314 18178 server_base.cc:1034] running on GCE node
I20260810 07:24:17.993753 18178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:17.994154 18178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:17.995333 18178 hybrid_clock.cc:648] HybridClock initialized: now 1786346657995315 us; error 42 us; skew 500 ppm
Aug 10 07:24:17 dist-test-slave-x5fp krb5kdc[18025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:17.998147 18178 init.cc:377] Logged in from keytab as kudu/127.14.164.61@KRBTEST.COM (short username kudu)
I20260810 07:24:17.998605 18178 webserver.cc:460] Webserver started at http://127.14.164.61:35635/ using document root <none> and password file <none>
I20260810 07:24:17.998852 18178 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:17.998941 18178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:17.999096 18178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:18.000234 18178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data/instance:
uuid: "bcb9b2e22d464e18b549497a03adba73"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-x5fp"
server_key: "ca7dc8deb49ba6ba946460e097495e41"
server_key_iv: "5e2299216e934747b38ad2443a1264a6"
server_key_version: "encryptionkey@0"
I20260810 07:24:18.000624 18178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal/instance:
uuid: "bcb9b2e22d464e18b549497a03adba73"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-x5fp"
server_key: "ca7dc8deb49ba6ba946460e097495e41"
server_key_iv: "5e2299216e934747b38ad2443a1264a6"
server_key_version: "encryptionkey@0"
I20260810 07:24:18.002094 18178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:24:18.002950 18197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:18.003134 18178 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:24:18.003248 18178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal
uuid: "bcb9b2e22d464e18b549497a03adba73"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-x5fp"
server_key: "ca7dc8deb49ba6ba946460e097495e41"
server_key_iv: "5e2299216e934747b38ad2443a1264a6"
server_key_version: "encryptionkey@0"
I20260810 07:24:18.003330 18178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:18.016010 18178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:18.018656 18178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:18.018847 18178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:18.023170 18178 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.61:39103
I20260810 07:24:18.023268 18275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.61:39103 every 8 connection(s)
I20260810 07:24:18.023644 18178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/data/info.pb
I20260810 07:24:18.024827 18276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:18.026500 18276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:18.029517 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18178
I20260810 07:24:18.029646 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-1/wal/instance
I20260810 07:24:18.029844 14992 external_mini_cluster.cc:1428] Setting key e057e2f49eb18c90be4e4acabd63746b
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:18.037781 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.027314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.14.164.62:46475)
0810 07:24:18.027449 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.027452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.027665 (+ 213us) server_negotiation.cc:407] Connection header received
0810 07:24:18.028269 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.028271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.028288 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.028343 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.029125 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.029975 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.031102 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.031490 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.031584 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.034467 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.034469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.034470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.034479 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.036199 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.036676 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.036678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.036679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.036791 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.037063 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.037064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.037064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.037194 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.037265 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.037583 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.037680 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 07:24:18.038004 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.027266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:46475 (local address 127.0.0.1:52060)
0810 07:24:18.027523 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.027565 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.028147 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.028435 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.028438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.028775 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.029015 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.029020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.030129 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.030130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.030990 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.030993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.031612 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.031613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.031813 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.032482 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.032494 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.034346 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.036302 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.036311 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.036315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.036590 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.036862 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.036863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.036864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.036986 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.037269 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.037277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.037446 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.037562 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.037735 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":118,"threads_started":1}
I20260810 07:24:18.038682 18276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.164.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.164.60@KRBTEST.COM" created.
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:24:18.047390 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.038996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50550)
0810 07:24:18.039238 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.039251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.039305 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.039455 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.039456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.039619 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.039671 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.039672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.040553 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.040554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.041166 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.041169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.041681 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.041681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.041749 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.042207 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.042212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.043525 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.045840 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.045841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.045842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.046096 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.046363 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.046364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.046364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.046469 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.046948 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.046949 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.047037 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.047127 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.047269 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:24:18.047492 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.039080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.14.164.61:39103)
0810 07:24:18.039259 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.039264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.039305 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:24:18.039353 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.039356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.039370 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.039454 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.039754 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.040458 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.041253 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.041602 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.041684 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.043633 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.043645 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.043645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.043655 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.045743 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.046214 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.046217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.046218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.046284 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.046600 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.046601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.046601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.046777 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.046864 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.047170 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.047360 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260810 07:24:18.048147 18276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:18.048588 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.048450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48862)
0810 07:24:18.048503 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.048557 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:24:18.048797 18200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
I20260810 07:24:18.049314 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.049215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48876)
0810 07:24:18.049267 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.049294 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:24:18.049439 18276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45893: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.164.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab.
Entry for principal kudu/127.14.164.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab.
I20260810 07:24:18.061543 14992 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMAuLWb/build/release/bin/kudu
/tmp/dist-test-taskMAuLWb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:45893
--webserver_interface=127.14.164.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab
--principal=kudu/127.14.164.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.164.20:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.164.62:46475,127.14.164.61:39103,127.14.164.60:45893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:18.078532 18169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } attempt: 3
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:24:18.087054 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.078911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50564)
0810 07:24:18.078993 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.079024 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.079087 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.079363 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.079364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.079533 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.079609 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.079611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.080650 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.080651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.081388 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.081390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.081979 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.081981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.082062 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.082647 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.082654 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.084013 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.085881 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.085882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.085882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.086081 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.086316 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.086317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.086317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.086393 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.086720 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.086722 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.086774 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.086844 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.086973 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:24:18.087081 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.079098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.14.164.61:39103)
0810 07:24:18.079175 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.079178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.079192 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:24:18.079225 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.079226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.079237 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.079316 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.079697 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.080537 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.081475 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.081820 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.081900 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.084131 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.084132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.084133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.084141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.085785 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.086158 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.086159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.086159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.086229 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.086459 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.086460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.086460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.086621 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.086692 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.086886 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.086988 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:24:18.087733 18169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:18.088147 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.088013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48878)
0810 07:24:18.088082 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.088119 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:24:18.088521 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.088415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48892)
0810 07:24:18.088472 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.088499 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 07:24:18.088634 18169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45893: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
I20260810 07:24:18.098652 18276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } attempt: 1
I20260810 07:24:18.099354 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.099177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48896)
0810 07:24:18.099257 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.099312 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:24:18.099917 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.099759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48906)
0810 07:24:18.099827 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.099855 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 07:24:18.100075 18276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45893: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
I20260810 07:24:18.119827 18169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } attempt: 1
I20260810 07:24:18.120465 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.120286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48922)
0810 07:24:18.120361 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.120420 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:24:18.120996 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.120876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48936)
0810 07:24:18.120929 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.120969 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:24:18.121184 18169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45893: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
W20260810 07:24:18.155062 18285 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:18.155334 18285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:18.155355 18285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:18.155380 18285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:18.156653 18285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:18.156769 18285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:18.156807 18285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:18.156827 18285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:18.156850 18285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:18.158481 18285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.164.20:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.164.62:46475,127.14.164.61:39103,127.14.164.60:45893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/krb5kdc/kudu_127.14.164.60.keytab
--principal=kudu/127.14.164.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.164.60:45893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.164.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:24:18.158725 18285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:18.158937 18285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:18.164355 18299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:18.164358 18306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:18.164590 18285 server_base.cc:1034] running on GCE node
W20260810 07:24:18.164372 18301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:18.164968 18285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:18.165289 18285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:18.166463 18285 hybrid_clock.cc:648] HybridClock initialized: now 1786346658166460 us; error 40 us; skew 500 ppm
I20260810 07:24:18.168232 18276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } attempt: 2
I20260810 07:24:18.168741 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.168575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48948)
0810 07:24:18.168629 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.168670 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346658, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:18.169117 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.168998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48962)
0810 07:24:18.169053 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.169087 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:24:18.169221 18276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45893: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
I20260810 07:24:18.169504 18285 init.cc:377] Logged in from keytab as kudu/127.14.164.60@KRBTEST.COM (short username kudu)
I20260810 07:24:18.170014 18285 webserver.cc:460] Webserver started at http://127.14.164.60:41793/ using document root <none> and password file <none>
I20260810 07:24:18.170270 18285 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:18.170317 18285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:18.170490 18285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:18.171684 18285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data/instance:
uuid: "e109404487954cd298f67b106e853fa3"
format_stamp: "Formatted at 2026-08-10 07:24:18 on dist-test-slave-x5fp"
server_key: "10ba993e88808d007ef884a1ec079631"
server_key_iv: "be312021178a1864231da9db41e62e78"
server_key_version: "encryptionkey@0"
I20260810 07:24:18.172094 18285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal/instance:
uuid: "e109404487954cd298f67b106e853fa3"
format_stamp: "Formatted at 2026-08-10 07:24:18 on dist-test-slave-x5fp"
server_key: "10ba993e88808d007ef884a1ec079631"
server_key_iv: "be312021178a1864231da9db41e62e78"
server_key_version: "encryptionkey@0"
I20260810 07:24:18.173786 18285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:24:18.174759 18320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:18.175001 18285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:24:18.175103 18285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data,/tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal
uuid: "e109404487954cd298f67b106e853fa3"
format_stamp: "Formatted at 2026-08-10 07:24:18 on dist-test-slave-x5fp"
server_key: "10ba993e88808d007ef884a1ec079631"
server_key_iv: "be312021178a1864231da9db41e62e78"
server_key_version: "encryptionkey@0"
I20260810 07:24:18.175164 18285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:18.196405 18169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } attempt: 2
I20260810 07:24:18.196969 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.196832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48970)
0810 07:24:18.196895 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.196936 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:24:18.197391 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.197243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:48984)
0810 07:24:18.197308 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.197354 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:24:18.197539 18169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.164.60:45893: Network error: Client connection negotiation failed: client connection to 127.14.164.60:45893: connect: Connection refused (error 111)
I20260810 07:24:18.210520 18285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:18.213320 18285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:18.213503 18285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:18.218443 18285 rpc_server.cc:307] RPC server started. Bound to: 127.14.164.60:45893
I20260810 07:24:18.218644 18395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.164.60:45893 every 8 connection(s)
I20260810 07:24:18.218978 18285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/data/info.pb
I20260810 07:24:18.220139 18396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:18.222141 18396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346658, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
I20260810 07:24:18.228904 14992 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMAuLWb/build/release/bin/kudu as pid 18285
I20260810 07:24:18.229027 14992 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMAuLWb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346363752364-14992-0/minicluster-data/master-2/wal/instance
I20260810 07:24:18.229267 14992 external_mini_cluster.cc:1428] Setting key 3a90b314a2aaa72a54d2ae8bc62dbc1b
I20260810 07:24:18.233825 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.223077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.14.164.62:46475)
0810 07:24:18.223154 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.223156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.223415 (+ 259us) server_negotiation.cc:407] Connection header received
0810 07:24:18.223952 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.223953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.223965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.224044 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.224971 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.225461 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.226592 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.226792 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.226863 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.229610 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.229612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.229612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.229624 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.232151 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.232632 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.232634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.232634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.232709 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.233042 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.233043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.233044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.233209 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.233263 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.233614 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.233720 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:24:18.234377 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.222969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.62:46475 (local address 127.0.0.1:52062)
0810 07:24:18.223231 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.223287 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.223801 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.224204 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.224208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.224496 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.224803 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.224816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.225570 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.225571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.226437 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.226440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.226928 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.226930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.227071 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.227670 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.227681 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.229498 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.232253 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.232261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.232263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.232546 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.232827 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.232828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.232829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.232958 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.233286 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.233291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.233463 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.233590 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.233779 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":142,"threads_started":1}
I20260810 07:24:18.235111 18396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346658, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:24:18.240332 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.229723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.14.164.60:45893)
0810 07:24:18.229952 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.229958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.229988 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:24:18.230048 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.230050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.230062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.230126 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.230925 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.232159 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.233158 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.233639 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.233704 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.236695 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.236705 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.236705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.236713 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.238799 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.239345 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.239349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.239350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.239423 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.239630 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.239631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.239631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.239830 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.240073 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.240123 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.240214 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":121,"threads_started":1}
I20260810 07:24:18.243635 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.235414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.61:39103 (local address 127.0.0.1:50570)
0810 07:24:18.236200 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.236248 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.236308 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.236456 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.236457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.236582 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.236666 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.236668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.237423 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.237424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.238242 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.238245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.238626 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.238627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.238725 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.239050 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.239055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.240513 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.242560 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.242561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.242562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.242739 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.242960 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.242961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.242962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.243021 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.243297 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.243299 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.243344 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.243429 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.243535 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765}
I20260810 07:24:18.243669 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.235493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50570 (local address 127.14.164.61:39103)
0810 07:24:18.235545 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.235547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.236315 (+ 768us) server_negotiation.cc:407] Connection header received
0810 07:24:18.236358 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.236359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.236374 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.236448 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.236816 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.237338 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.238336 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.238543 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.238602 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.240608 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.240609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.240610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.240619 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.242475 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.242825 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.242826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.242826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.242891 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.243097 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.243098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.243098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.243221 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.243295 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.243498 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.243578 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:24:18.244002 18396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.62@KRBTEST.COM
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346658, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.60@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:18.249630 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.241376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.14.164.62:46475)
0810 07:24:18.241415 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.241417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.241425 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:24:18.241449 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.241450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.241463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.241518 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.242204 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.242709 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.243463 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.243670 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.243747 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.246309 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.246311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.246311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.246320 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.248204 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.248636 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.248637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.248637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.248700 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.249037 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.249039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.249039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.249173 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.249390 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.249430 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.249515 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 07:24:18.251575 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.244296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:49000)
0810 07:24:18.244339 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.244352 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.244395 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.244503 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.244504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.244642 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.244671 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.244672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.245364 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.245365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.245909 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.245912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.246415 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.246416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.246487 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.247042 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.247049 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.248441 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.250363 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.250363 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.250364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.250635 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.250858 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.250859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.250859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.250962 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.251227 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.251228 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.251268 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.251350 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.251457 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 07:24:18.251626 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.244361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.14.164.60:45893)
0810 07:24:18.244394 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.244396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.244412 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:24:18.244437 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.244438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.244448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.244493 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.244762 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.245274 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.245997 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.246321 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.246374 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.248537 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.248538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.248539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.248546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.250257 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.250727 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.250728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.250728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.250789 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.251030 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.251031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.251031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.251149 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.251223 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.251374 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.251507 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":19}
I20260810 07:24:18.253227 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: Bootstrap starting.
I20260810 07:24:18.254314 18396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:18.254645 18396 log.cc:824] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.164.61@KRBTEST.COM
I20260810 07:24:18.255810 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: No bootstrap required, opened a new log
I20260810 07:24:18.257781 18396 raft_consensus.cc:348] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.257941 18396 raft_consensus.cc:729] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e109404487954cd298f67b106e853fa3, State: Initialized, Role: FOLLOWER
I20260810 07:24:18.258051 18396 consensus_queue.cc:260] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.258446 18396 sys_catalog.cc:564] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:18.259055 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.250519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.14.164.61:39103)
0810 07:24:18.250588 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.250590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.250599 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:24:18.250622 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.250623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.250633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.250703 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.251200 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.251742 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.252644 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.252985 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.253385 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.256153 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.256155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.256155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.256164 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.257764 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.258198 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.258200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.258200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.258270 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.258498 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.258499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.258500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.258638 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.258816 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.258865 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.258947 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:24:18.258389 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.259317 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:18.260347 18276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } attempt: 3
I20260810 07:24:18.260699 14992 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 07:24:18.261006 18425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:18.261089 18425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:18.261879 18125 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52064
I20260810 07:24:18.262023 18285 master_runner.cc:186] Error getting master registration for 127.14.164.62:46475: OK, instance_id { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" instance_seqno: 1786346657850519 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.61@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:18.265223 18285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:18.269188 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.260654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:49004)
0810 07:24:18.260768 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.260779 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.260820 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.261083 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.261084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.261263 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.261312 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.261314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.262113 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.262114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.262859 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.262862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.263485 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.263486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.263549 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.264057 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.264063 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.265597 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.267825 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.267826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.267827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.268090 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.268342 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.268343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.268343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.268430 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.268813 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.268814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.268874 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.268958 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.269080 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 07:24:18.269203 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.260918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.14.164.60:45893)
0810 07:24:18.260959 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.260961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.260969 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:24:18.260990 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.260991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.261001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.261077 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.261400 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.262009 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.263059 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.263388 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.263477 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.265702 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.265704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.265704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.265712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.267705 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.268180 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.268181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.268182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.268260 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.268499 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.268500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.268500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.268745 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.268823 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.268994 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.269095 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:24:18.271350 18276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: Bootstrap starting.
I20260810 07:24:18.272112 18276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:18.272475 18276 log.cc:824] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:18.273355 18276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: No bootstrap required, opened a new log
I20260810 07:24:18.274989 18276 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.275169 18276 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb9b2e22d464e18b549497a03adba73, State: Initialized, Role: FOLLOWER
I20260810 07:24:18.275266 18276 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.275594 18430 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.275681 18276 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:18.275735 18430 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:24:18.277684 18447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:18.277794 18447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:18.277930 18178 master_runner.cc:186] Error getting master registration for 127.14.164.62:46475: OK, instance_id { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" instance_seqno: 1786346657850519 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:18.281095 18178 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:18.296734 18169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } attempt: 3
Aug 10 07:24:18 dist-test-slave-x5fp krb5kdc[18025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.164.62@KRBTEST.COM for kudu/127.14.164.60@KRBTEST.COM
I20260810 07:24:18.305295 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.297227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.164.60:45893 (local address 127.0.0.1:49016)
0810 07:24:18.297300 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:18.297313 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:18.297363 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:18.297760 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:18.297761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:18.297955 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:18.298014 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.298016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.298971 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.298973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.299773 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:18.299776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.300160 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.300162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:18.300227 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.300790 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:18.300797 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:18.302401 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:18.304100 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.304101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.304102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.304334 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.304534 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:18.304535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:18.304536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:18.304600 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:18.304932 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:18.304933 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:18.305001 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 07:24:18.305067 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:24:18.305179 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 07:24:18.305655 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.297288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49016 (local address 127.14.164.60:45893)
0810 07:24:18.297334 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.297336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.297350 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:24:18.297644 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.297646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.297677 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.297729 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.298110 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.298851 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.299863 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.300071 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.300158 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.302506 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.302508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.302509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.302517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.303988 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.304402 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.304404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.304404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.304462 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.304655 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.304656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.304656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.304857 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.304925 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.305444 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.305539 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:24:18.307266 18169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f: Bootstrap starting.
I20260810 07:24:18.307953 18169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:18.308271 18169 log.cc:824] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:18.309406 18169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f: No bootstrap required, opened a new log
I20260810 07:24:18.310916 18169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.311089 18169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168c43fb9b7c41a8bc05fb2ef402a70f, State: Initialized, Role: FOLLOWER
I20260810 07:24:18.311193 18169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.311614 18125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:24:18.311559 18453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.311709 18125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:18.311770 18453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:18.312469 18125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.312793 18125 leader_election.cc:288] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [CANDIDATE]: Term 1 election: Requested vote from peers bcb9b2e22d464e18b549497a03adba73 (127.14.164.61:39103), e109404487954cd298f67b106e853fa3 (127.14.164.60:45893)
I20260810 07:24:18.313320 18240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bcb9b2e22d464e18b549497a03adba73"
I20260810 07:24:18.313494 18240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:18.314033 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e109404487954cd298f67b106e853fa3"
I20260810 07:24:18.314179 18359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:18.314572 18240 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 168c43fb9b7c41a8bc05fb2ef402a70f in term 1.
I20260810 07:24:18.314931 18076 leader_election.cc:302] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 168c43fb9b7c41a8bc05fb2ef402a70f, bcb9b2e22d464e18b549497a03adba73; no voters:
I20260810 07:24:18.315117 18453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:18.315215 18359 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 168c43fb9b7c41a8bc05fb2ef402a70f in term 1.
I20260810 07:24:18.311596 18169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:18.315558 18453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [term 1 LEADER]: Becoming Leader. State: Replica: 168c43fb9b7c41a8bc05fb2ef402a70f, State: Running, Role: LEADER
I20260810 07:24:18.315694 18453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } }
I20260810 07:24:18.316258 18453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 168c43fb9b7c41a8bc05fb2ef402a70f. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.316377 18453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:18.317200 18476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:18.318650 18476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:18.320446 18359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [term 1 FOLLOWER]: Refusing update from remote peer 168c43fb9b7c41a8bc05fb2ef402a70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:18.320533 18240 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [term 1 FOLLOWER]: Refusing update from remote peer 168c43fb9b7c41a8bc05fb2ef402a70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:18.320827 18453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:18.322898 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 168c43fb9b7c41a8bc05fb2ef402a70f. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.323015 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:18.323127 18453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:18.325587 18430 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 168c43fb9b7c41a8bc05fb2ef402a70f. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.325701 18430 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:18.326453 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.317101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50582 (local address 127.14.164.61:39103)
0810 07:24:18.317154 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.317156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.318161 (+ 1005us) server_negotiation.cc:407] Connection header received
0810 07:24:18.318308 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.318309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.318324 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.318359 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.319245 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.319988 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.321078 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.321345 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.321389 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.323205 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.323207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.323207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.323216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.325008 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.325479 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.325481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.325482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.325558 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.325779 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.325780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.325780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.325935 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.326021 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.326265 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.326354 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:24:18.331326 18500 mvcc.cc:205] Tried to move back new op lower bound from 7316875912478306304 to 7316875912462209024. Current Snapshot: MvccSnapshot[applied={T|T < 7316875912478306304}]
I20260810 07:24:18.331147 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.331635 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:18.331759 18461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.331897 18461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:18.332099 18498 mvcc.cc:205] Tried to move back new op lower bound from 7316875912478306304 to 7316875912462209024. Current Snapshot: MvccSnapshot[applied={T|T < 7316875912478306304}]
I20260810 07:24:18.335295 18499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.335462 18499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:18.336068 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.317529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.14.164.60:45893)
0810 07:24:18.317576 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.317578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.327057 (+ 9479us) server_negotiation.cc:407] Connection header received
0810 07:24:18.327157 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.327158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.327170 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.327200 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.327905 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.328622 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.330566 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.330851 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.330905 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.332706 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.332708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.332708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.332738 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.334606 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.335011 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.335013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.335013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.335078 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.335286 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.335287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.335288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.335438 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.335526 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.335762 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.335851 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:24:18.336239 18430 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168c43fb9b7c41a8bc05fb2ef402a70f" member_type: VOTER last_known_addr { host: "127.14.164.62" port: 46475 } } peers { permanent_uuid: "bcb9b2e22d464e18b549497a03adba73" member_type: VOTER last_known_addr { host: "127.14.164.61" port: 39103 } } peers { permanent_uuid: "e109404487954cd298f67b106e853fa3" member_type: VOTER last_known_addr { host: "127.14.164.60" port: 45893 } } }
I20260810 07:24:18.336344 18430 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:18.336932 18503 mvcc.cc:205] Tried to move back new op lower bound from 7316875912478306304 to 7316875912462209024. Current Snapshot: MvccSnapshot[applied={T|T < 7316875912478306304}]
I20260810 07:24:18.338475 18476 catalog_manager.cc:1347] Generated new cluster ID: 2c48ecb3c01b460ead1be5c0e1b7659e
I20260810 07:24:18.338533 18476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:18.352341 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:18.317788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.14.164.62:46475)
0810 07:24:18.317830 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.317832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.332344 (+ 14512us) server_negotiation.cc:407] Connection header received
0810 07:24:18.333446 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.333447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.333460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.333494 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.337082 (+ 3588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.337590 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.338297 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:18.338579 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:18.338645 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:18.349116 (+ 10471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:18.349118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:18.349118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:18.349128 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:18.350902 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.351365 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.351366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.351366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.351433 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:18.351622 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:18.351623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:18.351623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:18.351829 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:18.351924 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:18.352155 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 07:24:18.352239 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:24:18.359138 18476 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:18.359953 18476 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:18.375243 18476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 168c43fb9b7c41a8bc05fb2ef402a70f: Generated new TSK 0
I20260810 07:24:18.375600 18476 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:24:18.409315 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.407477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.14.164.61:39103)
0810 07:24:18.407530 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.407532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.408495 (+ 963us) server_negotiation.cc:407] Connection header received
0810 07:24:18.408610 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.408611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.408623 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.408654 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.409213 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50590: BlockingRecv error: recv got EOF from 127.0.0.1:50590 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260810 07:24:18.410149 18402 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.407924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49036 (local address 127.14.164.60:45893)
0810 07:24:18.407975 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.407977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.409604 (+ 1627us) server_negotiation.cc:407] Connection header received
0810 07:24:18.409705 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.409707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.409720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.409750 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.410085 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49036: BlockingRecv error: recv got EOF from 127.0.0.1:49036 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 07:24:18.411134 18172 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.408295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52086 (local address 127.14.164.62:46475)
0810 07:24:18.408340 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.408342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.410418 (+ 2076us) server_negotiation.cc:407] Connection header received
0810 07:24:18.410537 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.410539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.410551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.410582 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.411069 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52086: BlockingRecv error: recv got EOF from 127.0.0.1:52086 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:24:18.413537 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.412578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.14.164.61:39103)
0810 07:24:18.412620 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.412621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.412922 (+ 301us) server_negotiation.cc:407] Connection header received
0810 07:24:18.413057 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.413058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.413081 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.413112 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.413483 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50602: BlockingRecv error: recv got EOF from 127.0.0.1:50602 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 07:24:18.416946 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.416036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50618 (local address 127.14.164.61:39103)
0810 07:24:18.416075 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.416077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.416345 (+ 268us) server_negotiation.cc:407] Connection header received
0810 07:24:18.416448 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.416449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.416461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.416490 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.416886 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50618: BlockingRecv error: recv got EOF from 127.0.0.1:50618 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 07:24:18.421613 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.420454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.14.164.61:39103)
0810 07:24:18.420500 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.420502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.420822 (+ 320us) server_negotiation.cc:407] Connection header received
0810 07:24:18.420936 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.420937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.420951 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.420982 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.421539 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50620: BlockingRecv error: recv got EOF from 127.0.0.1:50620 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 07:24:18.427927 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.426978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.14.164.61:39103)
0810 07:24:18.427023 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.427025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.427310 (+ 285us) server_negotiation.cc:407] Connection header received
0810 07:24:18.427430 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.427431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.427444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.427475 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.427861 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50626: BlockingRecv error: recv got EOF from 127.0.0.1:50626 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:24:18.438504 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.437465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50640 (local address 127.14.164.61:39103)
0810 07:24:18.437518 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.437520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.437847 (+ 327us) server_negotiation.cc:407] Connection header received
0810 07:24:18.437952 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.437953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.437967 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.437998 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.438434 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50640: BlockingRecv error: recv got EOF from 127.0.0.1:50640 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 07:24:18.457073 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.455980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.14.164.61:39103)
0810 07:24:18.456029 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.456032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.456347 (+ 315us) server_negotiation.cc:407] Connection header received
0810 07:24:18.456480 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.456481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.456494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.456527 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.457003 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50652: BlockingRecv error: recv got EOF from 127.0.0.1:50652 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:24:18.491726 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.490632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.14.164.61:39103)
0810 07:24:18.490687 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.490690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.491047 (+ 357us) server_negotiation.cc:407] Connection header received
0810 07:24:18.491126 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.491127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.491143 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.491213 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.491645 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50666: BlockingRecv error: recv got EOF from 127.0.0.1:50666 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 07:24:18.557987 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.557142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.14.164.61:39103)
0810 07:24:18.557259 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.557262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.557350 (+ 88us) server_negotiation.cc:407] Connection header received
0810 07:24:18.557404 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.557405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.557417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.557478 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.557885 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50674: BlockingRecv error: recv got EOF from 127.0.0.1:50674 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 07:24:18.688097 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.687357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50688 (local address 127.14.164.61:39103)
0810 07:24:18.687453 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.687456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.687474 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:24:18.687552 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.687553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.687563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.687625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.688003 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50688: BlockingRecv error: recv got EOF from 127.0.0.1:50688 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 07:24:18.946755 18280 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:18.945714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.14.164.61:39103)
0810 07:24:18.945842 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:18.945847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:18.945867 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:24:18.945903 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:18.945904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:18.945916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:18.945992 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:18.946682 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50690: BlockingRecv error: recv got EOF from 127.0.0.1:50690 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 07:24:19.261708 18425 catalog_manager.cc:1259] Loaded cluster ID: 2c48ecb3c01b460ead1be5c0e1b7659e
I20260810 07:24:19.261811 18425 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: loading cluster ID for follower catalog manager: success
I20260810 07:24:19.262676 18425 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: acquiring CA information for follower catalog manager: success
I20260810 07:24:19.262974 18425 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e109404487954cd298f67b106e853fa3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:19.278431 18447 catalog_manager.cc:1259] Loaded cluster ID: 2c48ecb3c01b460ead1be5c0e1b7659e
I20260810 07:24:19.278530 18447 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: loading cluster ID for follower catalog manager: success
I20260810 07:24:19.279325 18447 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: acquiring CA information for follower catalog manager: success
I20260810 07:24:19.279584 18447 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bcb9b2e22d464e18b549497a03adba73: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:19.463327 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.460421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.14.164.61:39103)
0810 07:24:19.460582 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.460584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.460625 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:24:19.460751 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.460752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.460821 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.460903 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:19.461162 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.461670 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.462471 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.462669 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.462729 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:19.462940 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:19.463071 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:19.463136 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.463188 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.463205 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260810 07:24:19.468132 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.465618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.14.164.60:45893)
0810 07:24:19.465731 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.465733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.465742 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:24:19.465810 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.465811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.465849 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.465922 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:19.466060 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.466542 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.467278 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.467479 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.467537 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:19.467710 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:19.467830 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:19.467901 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.467965 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.467985 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260810 07:24:19.473300 18623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.470145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52102 (local address 127.14.164.62:46475)
0810 07:24:19.470258 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.470261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.470668 (+ 407us) server_negotiation.cc:407] Connection header received
0810 07:24:19.470722 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.470724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.470775 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.470851 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:19.471029 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.471536 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.472387 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.472634 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.472709 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:19.472950 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:19.473041 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:19.473122 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.473180 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.473203 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":59,"threads_started":1}
I20260810 07:24:19.473647 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.470142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.14.164.61:39103)
0810 07:24:19.470201 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.470203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.470399 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:24:19.470506 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.470507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.470521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.470551 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:19.470736 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.471506 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.472659 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.472981 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.473064 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:19.473352 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:19.473389 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:19.473419 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.473538 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.473565 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:24:19.474038 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.470581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.14.164.60:45893)
0810 07:24:19.470620 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.470622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.470788 (+ 166us) server_negotiation.cc:407] Connection header received
0810 07:24:19.470846 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.470847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.470857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.470910 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:19.471063 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.471782 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.472844 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:19.473178 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:19.473237 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:19.473436 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:19.473468 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:19.473498 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.473916 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.473945 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:24:19.475601 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18040
I20260810 07:24:19.486697 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18178
I20260810 07:24:19.497675 14992 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMAuLWb/build/release/bin/kudu with pid 18285
2026-08-10T07:24:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3237 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6440 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (295758 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST